Issues with Eye Candy - For example overly intense or window darkness after cast

[bluap]

In the 1.9.5p9 release forum thread, a number of people are reporting issues with Eye Candy effects.  Either with loosing light after an MD effects or seeing extra intense MD effects.  Please could people that have these or similar issues with Eye Candy effects please provide details of their Eye Candy settings, OS, Graphics card and drivers etc.  If you think you have an idea of when this first started happening, please say.  This could be after a particular client update, or an OS/driver update.  If you really don't know, please just say.

 

I'm not seeing the issue but the details for me this are:

Max Effects Framerate: 45.0

Min Effects Framerate: 5:0

Light columns threshold: 5

Max Idle Cycles Per Second: 40

FPS: 60

Video card: NVIDIA GeForce GTX 750 Ti/PCIe/SSE2
Vendor ID: NVIDIA Corporation
OpenGL Version: 4.6.0 NVIDIA 470.57.02

Client Version: Latest Git

OS: Linux Ubuntu 21.04

[Aislinn]

The two eye candy effects I've observed are:

1. The particle columns that happen after I kill a mob occasionally are bigger and brighter than they usually are.
2. The mana drain spell effect is occasionally bigger and brighter than it usually is.

Both can happen if I cast the spell or kill the mob, or if others do also.  Doesn't seem to be a distinction there.  The entire screen doesn't change, just the specific eye candy event does.  It may happen on other things that I just haven't paid attention to, or it may not.  No idea.  I have no idea when this started other than it's been happening for quite some time and I just thought it was supposed to.  I did speculate once in a while on various reasons somebody might have coded it this way, like the mob was special in some way or something.  As far as I can tell, nothing "bad" happens (for me) like client crashing, or going blank or bright or black.  I just pegged it as one of those EL quirks that nobody ever explained.  Then one day during a recent invasion somebody casually mentioned it and others (including me) said "Yeah, that happens to me too!" so I posted.

 

My details:

Max Effects Framerate: 45.00

Min Effects Framerate: 12.00

Light columns threshold: 2

Max Idle Cycles Per Second: 32

FPS: Limited to 45

Video card: NVIDIA GeForce GTX 1050 Ti/PCIe/SSE2
Vendor ID: NVIDIA Corporation
OpenGL Version: 4.6.0 NVIDIA 470.63.01

Client Version: Latest Git

OS: Linux arch 5.13.9-arch1-1

 

I guess I should add I have no idea what most of these settings do or what they should be set at (other than FPS).  I don't know what default is, or remember if I've tinkered with them or not.

[Nogrod]

1. Often, when fighting in invasions, if someone casts mana drain in my vicinity, my screen goes black until the spell effect has completed.
   1. I definitely saw this before the 1.9.5p9 release but I don't know when I first started seeing it, probably in the last 3 years and I can't say if it was related to a client update or a system update..
2. When killing an invasion or summoning mob, sometimes the particle column effect is very bright but I noticed that it only happens when someone kills a mob with the particle column being between directly in front of the scene camera and so while it's very bright, it does not seem unnatural. To recreate this, have someone summon a mob and kill it while they're just under your game window scene so that the particle column is directly below your character and have the camera tilted so that your looking mostly down the column. I can't say that this is the same thing that others are seeing.

 

 

My details:

Max Effects Framerate: 45.00

Min Effects Framerate: 15.00

Light columns threshold: 5

Max Idle Cycles Per Second: 40

FPS: Limited to 40

Video card: VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Juniper PRO [Radeon HD 6750]

client glx vendor string: Mesa Project and SGI
client glx version string: 1.4

OpenGL vendor string: X.Org
OpenGL renderer string: AMD JUNIPER (DRM 2.50.0 / 5.12.15-arch1-1, LLVM 12.0.0)

OpenGL version string: 3.1 Mesa 21.1.4

Client Version: Latest Git

OS: Arch Linux 5.12.15-arch1-1

 

 

 

 

[bluap]

Thanks both for the info, nothing obvious jumps out.  Have either of you played around with the Eye candy test window?  If you enabled ECDEBUGWIN and rebuild, you can open the test window using ctrl+alt+c.  You can then generate EC effects without needing to cast the spells.  Obviously, only you see them.  It would be interesting if you experience the problem using this method.  Perhaps even build with debug and the ADDRSANITISER library option or using valgrind.

 

After editing CMakeLists.txt to uncomment "-DECDEBUGWIN", to build with debug and the ADDRSANITISER library option using CMAKE you can something like this in the source directory:

cmake -B debug-build -DCMAKE_BUILD_TYPE=debug -DADDRSANITISER=On
make -C debug-build/
# assuming you have #data_dir properly defined in el.ini.....
debug-build/el.linux.bin test

 

[Raz]

I still get this - happened earlier this week one within 5mins of login - at that point I had not seen any other players - so just me fighting & casting heal / MD.

 

Deliberately I have not quit out from that session - and it has steadily got worse.  Now it pretty much happens every time I or anyone in view casts MD spell.

 

Sadly I'm a newbie windows user - not a clue how to build my own client - SRY!

[Grum]

34 minutes ago, Raz said:

at that point I had not seen any other players

That in itself is useful information, thanks.

35 minutes ago, Raz said:

and it has steadily got worse.  Now it pretty much happens every time I or anyone in view casts MD spell.

And that too. So it happens not just when you cast it yourself, also when someone else does.

 

What graphics card are you using? (#glinfo can tell you in game)

[Raz]

16 minutes ago, Grum said:

What graphics card are you using? (#glinfo can tell you in game)

[19:21:14] Video card: Intel(R) UHD Graphics
[19:21:14] Vendor ID: Intel
[19:21:14] OpenGL Version: 4.6.0 - Build 27.20.100.9565
 

Startup Gra[hocs messages:

 

[19:21:50] GL_ARB_multitexture extension found, using it.
[19:21:50] GL_ARB_texture_env_combine extension found, using it.
[19:21:50] GL_EXT_compiled_vertex_array extension found, using it.
[19:21:50] GL_ARB_point_sprite extension found, using it.
[19:21:50] GL_ARB_texture_compression extension found, using it.
[19:21:50] GL_EXT_texture_compression_s3tc extension found, using it.
[19:21:50] GL_SGIS_generate_mipmap extension found, using it.
[19:21:50] GL_ARB_shadow extension found, using it.
[19:21:50] GL_ARB_vertex_buffer_object extension found, using it.
[19:21:50] GL_EXT_framebuffer_object extension found, using it.
[19:21:50] GL_EXT_draw_range_elements extension found, using it.
[19:21:50] GL_ARB_texture_non_power_of_two extension found, using it.
[19:21:50] GL_ARB_fragment_program extension found, using it.
[19:21:50] GL_ARB_vertex_program extension found, using it.
[19:21:50] GL_ARB_fragment_shader extension found, using it.
[19:21:50] GL_ARB_vertex_shader extension found, using it.
[19:21:50] GL_ARB_shader_objects extension found, using it.
[19:21:50] GL_ARB_shading_language_100 extension found, using it.
[19:21:50] GL_ARB_texture_mirrored_repeat extension found, NOT using it...
[19:21:50] GL_ARB_texture_rectangle extension found, NOT using it...
[19:21:50] GL_EXT_fog_coord extension found, NOT using it...
[19:21:50] Couldn't find the GL_ATI_texture_compression_3dc extension, not using it...
[19:21:50] Couldn't find the GL_EXT_texture_compression_latc extension, not using it...
[19:21:50] GL_EXT_texture_filter_anisotropic extension found, using it.
[19:21:51] Not using vertex program for actor animation.
 

[Nogrod]

On 9/6/2021 at 6:27 PM, bluap said:

Thanks both for the info, nothing obvious jumps out.  Have either of you played around with the Eye candy test window?  If you enabled ECDEBUGWIN and rebuild, you can open the test window using ctrl+alt+c.  You can then generate EC effects without needing to cast the spells.  Obviously, only you see them.  It would be interesting if you experience the problem using this method.  Perhaps even build with debug and the ADDRSANITISER library option or using valgrind.

 

After editing CMakeLists.txt to uncomment "-DECDEBUGWIN", to build with debug and the ADDRSANITISER library option using CMAKE you can something like this in the source directory:

cmake -B debug-build -DCMAKE_BUILD_TYPE=debug -DADDRSANITISER=On
make -C debug-build/
# assuming you have #data_dir properly defined in el.ini.....
debug-build/el.linux.bin test

 

 

Just an update on trying to debug this issue. I've been slow to respond because I haven't been able to do many invasions lately and so I haven't seen the issue all that often.

 

I enabled both the DECDEBUGWIN and ADDRSANITISER options.

Unfortunately ADDRSANITISER is showing nothing.

The special effects debug window is nice but has been of limited use for me because of the following:

 

I only see this issue when there is a certain density of mobs on the screen.  I don't know what that number is.  If there are 20 mobs on the screen, I don't see the issue. If the invasion is packed, I can expect to see the issue.  It seems that the number of mobs triggering this varies and I wonder if it has something to do with load on my system.

 

Not all my system have this problem.  I have a more powerful System 76 Laptop running Ubuntu with an Intel(R) UHD Graphics 630 (CFL GT2) card.  It does not show the issue.

 

 

 

[Raz]

Update:

Twice during wvf invasion - I had the black screen effect immediately on login.  No other players were around - just me a load of mobs & a teleport pad.... could the additional particles in that contribute?

 

I did manager to recreate once - ie logout, re open client - cast MD and get black screen.

I then quit that client - quit all other clients and login again - and it was no longer an issue.  Think unrelated.

[Grum]

29 minutes ago, Raz said:

could the additional particles in that contribute?

It is likely. My gut feeling is that these issues are somehow caused by the eye candy effects, but I have read the code over a few times and have found nothing that stands out. From what I read in this thread, it also seems to build up somehow, with black screens becoming more common with more effects, actors, clients being involved. This hints at a memory corruption or other resource issue.

 

Then again, my gut might be totally wrong in this, I don't know.

 

It would help if I could somehow replicate it, but so far I have not been able to. Using integrated Intel graphics on Linux btw:

Video card: Mesa Intel(R) HD Graphics 530 (SKL GT2)
Vendor ID: Intel
OpenGL Version: 4.6 (Compatibility Profile) Mesa 21.2.1

 

Edited October 18, 2021 by Grum

[Burn]

On 10/11/2021 at 3:21 PM, Nogrod said:

I only see this issue when there is a certain density of mobs on the screen.  I don't know what that number is.  If there are 20 mobs on the screen, I don't see the issue. If the invasion is packed, I can expect to see the issue.  It seems that the number of mobs triggering this varies and I wonder if it has something to do with load on my system.

 

 

 

 

I've just invaded about 2k bears in DP crystal caves on Test server. They are capped at 1 so anyone can safely walk in there. Given the cave size that'll satisfy the heavy mob density requirement. My map testing is done so there's no reason for radu to restart it killing the invasion now.

 

If you have a char on Test that can do MD (the chars there are very, very old) or maybe someone else can go with you and MD near you, the bears can still be MD'ed despite the cap so that can be tested on them.

 

Just note that the bear will attack if you MD it, despite the cap.

 

ANYONE with a character old enough to be able to MD from the 1.9.3 client days should try to head to Test server with the last bluap client build and try this.

 

I've been keeping up making my own builds from the git code, been in many invasions and invances with it, and tested some of these bears as well. Haven't seen the issue on my part. (Linux on Intel with NVIDIA GeForce GTX 660 with latest drivers)

 

 

[Map note: Should be okay from IP to DP CC, but the maps on Test server are for 1.9.6, so you may find issues with mapwalking, or you drop into "holes" in random places, usually along edges of mountains/rocks/buildings. Don't be concerned with these, they're expected with the 1.9.5 client. ALWAYS CARRRY TELE ESSIES OR A RING IN CASE OF EMERGENCY.]

Edited October 20, 2021 by Burn

[bluap]

Just visited the cave and I've been able to reproduce the screen going black during the MD spell; this was running the client on my Raspberry Pi 400.  I'll investigate further but its a good step.

[Burn]

Ooops, more heavy of a test showed I was getting something.

 

This was actually harder to notice, but I seem to be getting the opposite effect. The lighting in the cavern gets really bright then goes back to normal.

 

Doesn't happen every time, but regularly.

 

 

Edited October 20, 2021 by Burn

[bluap]

A little more debugging on my Raspberry Pi: I'm seeing the screen going black when using the eye candy debug window so can save on essence.  I'm seeing the effect when using the Teleport-to-Range and Life-Drain effects too.  It also happens outside with a single creature.  TTR is particularly repeatable.  It's not consistent between runs though, perhaps suggesting uninitialized memory as the cause.

[Raz]

OMG thank you so much guys for making progress on this!

 

This sort of bug is a real pain to find - so much appreciated

[Grum]

 

19 hours ago, bluap said:

Just visited the cave and I've been able to reproduce the screen going black during the MD spell; this was running the client on my Raspberry Pi 400.  I'll investigate further but its a good step.

Great news! I might have to get a second Pi. (the first is tied up as a home theater system).

14 hours ago, bluap said:

A little more debugging on my Raspberry Pi: I'm seeing the screen going black when using the eye candy debug window so can save on essence.  I'm seeing the effect when using the Teleport-to-Range and Life-Drain effects too.  It also happens outside with a single creature.  TTR is particularly repeatable.  It's not consistent between runs though, perhaps suggesting uninitialized memory as the cause.

Might something be overwriting the global lighting level? What variables control this? Then again, the level goes back to normal. I'll do some digging.

[Grum]

14 hours ago, bluap said:

It also happens outside with a single creature.

In day light? If, as I suspect, it is a lighting issue, I would expect it to only occur on inside maps, or in night time.

[Grum]

1 hour ago, Grum said:

In day light? If, as I suspect, it is a lighting issue, I would expect it to only occur on inside maps, or in night time.

It probably is related to OpenGL lighting.

 

I managed to trigger the blackout on an old laptop (AMD graphics) by repeatedly ramming on TP to R in the eye candy debug window in the DP cave on the test server. Patching the code to not use lights in the eye candy effects:

diff --git a/eye_candy/eye_candy.cpp b/eye_candy/eye_candy.cpp
index d30c3267..0a1f01ca 100644
--- a/eye_candy/eye_candy.cpp
+++ b/eye_candy/eye_candy.cpp
@@ -1987,7 +1987,7 @@ namespace ec
 
                end_draw();
                // Draw lights.
-               if ((use_lights) && (particles.size() > 0))
+               if (false && (use_lights) && (particles.size() > 0))
                {
                        while (light_particles.size() < lights.size())
                                light_particles.push_back(std::pair<Particle*, float>(particles[randint((int)particles.size())], 0.0));

fixes hides the issue.

 

I still haven't found out why it blacks out though. AFAICT, the eye candy code uses at most 3 lights (out of a maximum of 8), and these are used by the EC code only.

Edited October 21, 2021 by Grum

[Grum]

It looks like invalid light color values (> 1) in the eye candy effects may be the root cause of the screen blacking on certain systems. I have checked in a patch on Github to fix this issue. I am not 100% sure this fixes the issue, as I am not always able to trigger the bug. Some session it triggers on the first TPTR, other times I cannot trigger it at all. I can confirm though, that with the fix I made, I have so far not been able to trigger it at all anymore.

 

If people who experience the screen blackening, and who are able to compile their own client, could test the fix, this would be much appreciated.

[Raz]

Thx again - sadly I'm a lazy windows user  But the idea of only happening at night (or insides) could well be relevant!  Will pay more attention if it happens again!

[bluap]

58 minutes ago, Grum said:

It looks like invalid light color values (> 1) in the eye candy effects may be the root cause of the screen blacking on certain systems. I have checked in a patch on Github to fix this issue. I am not 100% sure this fixes the issue, as I am not always able to trigger the bug. Some session it triggers on the first TPTR, other times I cannot trigger it at all. I can confirm though, that with the fix I made, I have so far not been able to trigger it at all anymore.

 

If people who experience the screen blackening, and who are able to compile their own client, could test the fix, this would be much appreciated.

Unfortunately, I'm still seeing the issue after the patch.  I've also been looking at the lights code but my time is limited currently, there's a great deal we could clean up there I think.  One thing I've yet to try is to adjust the randomness elements of the eye candy effects to made things a bit more repeatable for debugging.  Also, we've seen bugs with the eye candy code previously due to extremely large or small values, this could be another reason why its difficult to reproduce reliably.

[Grum]

Well, drats. That is most unfortunate, as after the patch I seem to be unable to trigger the issue again. And also, I'm clean out of ideas (unless the position of the light also does something crazy). If you find time though, could you double check on the Pi that the light color values are now all between 0 and 1? Perhaps there's an FP issue somewhere leading to negative or NaN values? Dunno, grasping at straws here.

 

W.r.t. cleaning up the lights code: my preferred method would be to use a proper shader and ditch the fixed mode lighting altogether. That might be a bit of work, though..

[Kaddy]

This happened to me in last Ida mines invasion. It was dark inside and when I md'ed, I had all light did get so bright.

 

My details:

Max Effects Framerate: 50.00

Min Effects Framerate: 10.00

Light columns threshold: 5

Max Idle Cycles Per Second: 60

FPS: Limited to 70

Video card: GeForce RTX 2060/PCIe/SSE2
Vendor ID: NVIDIA Corporation
OpenGL Version: 4.6.0 NVIDIA 446.14
Client Version: Eternal Lands Version 1.9.5p9

[bluap]

On 21/10/2021 at 2:00 PM, Grum said:

Well, drats. That is most unfortunate, as after the patch I seem to be unable to trigger the issue again. And also, I'm clean out of ideas (unless the position of the light also does something crazy). If you find time though, could you double check on the Pi that the light color values are now all between 0 and 1? Perhaps there's an FP issue somewhere leading to negative or NaN values? Dunno, grasping at straws here.

 

W.r.t. cleaning up the lights code: my preferred method would be to use a proper shader and ditch the fixed mode lighting altogether. That might be a bit of work, though..

 

Nothing odd about the brightness, no sign of NaNs.

 

However, I think have a work around for the black screen problem.  In lights.c init_lights() change the ec_add_light() calls to use lights greater than 8.  This could be done conditionally based on GL_MAX_LIGHT which is way greater than 8 on all my systems but the minimum is 8 according to the documentation so it might not work on all systems.  I've hacked being able to change these lights without restarting the client and I can turn the problem on and off in this way so it looks like a fix  of sorts.

 

However, it is only a work around and something is clearly messing with the lights.  I've looked though the code and found a possible bug in draw_lights() where GL_LIGHT4 is used sometimes but fixing this does not fix the problem so it could be down to more than one bug.  I'll carry on looking....

 

I think all my change was doing was just disabling the lights for eye candy.  Perhaps that's the solution, add a "trouble shooting" option to disable lights use with eye candy.

Edited October 24, 2021 by bluap

[bluap]

Cautiously optimistic that the problem is due to invalid particle positions , I'm just looking for the underlying cause or if I can just trap the values.