Jump to content


Photo

Map load times and map/3dobject compression


  • Please log in to reply
13 replies to this topic

#1 Learner

Learner

    Ant

  • Admin
  • 5430 posts
  • Gender:Male
  • Location:MN, USA
  • Interests:Persistent RPG

Posted 11 February 2007 - 08:49 PM

Map load times have been improved in the current CVS without any compile options.

Also, there is an optional map & 3D object compression that has recently been checked in that requires -DZLIB and ( -lz or -lzlib ) if not already being linked in. With this option enabled you can gzip compress all .elm and .e3d files (it's pk to leave the extensions of .elm.gz & .e3d.gz).

While most windows users won't have gzip available, they can use the gzip compressed files from Linux usiers.

Please test & post any comments on both of these speed ups.

#2 LabRat

LabRat

    Phantom Warrior

  • Members
  • 2620 posts
  • Gender:Male
  • Location:West Midlands, England, UK
  • Interests:mainly keeping Michelle_F off my ubercool one of a kind *autographed* EL T-shirt

Posted 11 February 2007 - 10:12 PM

I have tried the new CVS and I have to say that the loading times have decreased dramatically, from an estimated 6-7 seconds for the WS map originally to less than 2 with .e3d.gz and .elm.gz

Can this thread be deleted and the devs keep this for ourselves? :P

Here is what happened to the directory sizes:
Vanilla 1.3.3 134 MB (141,468,659 bytes)
gzip+libs+cvs 59.8 MB (62,769,155 bytes)

gzip+libs+cvs installer (just a rar.exe) 37.4 MB (39,279,033 bytes)
vanilla installer 29.1 MB (30,573,459 bytes)

more than a 50% reduction in directory size just using the .gz seems great to me, couple that with a massive load improvement and we have a definite winner \o/

#3 Kindar Naar

Kindar Naar

    Beaver

  • Members
  • 98 posts

Posted 11 February 2007 - 10:47 PM

more than a 50% reduction in directory size just using the .gz seems great to me, couple that with a massive load improvement and we have a definite winner \o/


Yup, I agree. I haven't done any benchmarks, but the improvement in load speed is clearly visible :P Great work :)

#4 crusadingknight

crusadingknight

    Feran

  • Members
  • 2103 posts
  • Gender:Male
  • Location:/usr/local/bin/rc, Belnan
  • Interests:Eternal Lands, AI Programming, D/C/Ruby Programming, Game and Integrated Interpreter Programming.

Posted 11 February 2007 - 11:02 PM

With so many e3ds to load, and most being within the size of a single DMA push, the major slowdown for them should be seek time - blobbing them into a common random-access archive format along with their textures might be better than having to do a seek for each one, since few are large enough anyway that individual compression helps.

Really great speedups though, I must say, most of all in the .elm department.

Edited by crusadingknight, 11 February 2007 - 11:03 PM.


#5 Learner

Learner

    Ant

  • Admin
  • 5430 posts
  • Gender:Male
  • Location:MN, USA
  • Interests:Persistent RPG

Posted 11 February 2007 - 11:29 PM

With so many e3ds to load, and most being within the size of a single DMA push, the major slowdown for them should be seek time - blobbing them into a common random-access archive format along with their textures might be better than having to do a seek for each one, since few are large enough anyway that individual compression helps.

Really great speedups though, I must say, most of all in the .elm department.

some E3D's are huge, but most of them are small. The patch allows reading either gzipped or ungzipped files easily. But another place where it wins is that the OS caching can now store more files as well.

Yes, blobbing them into a single file or group of files would help a little more, but not needed currently, maybe later :P

#6 bluap

bluap

    Feran

  • Moderators
  • 2215 posts
  • Gender:Male
  • Location:UK

Posted 12 February 2007 - 12:37 AM

Nice change :) A few quick tests suggests it could be a ~30% reduction for me for initially loading the game and something similar for moving from IP->WS the first time :P

#7 Espresso_Boy

Espresso_Boy

    Skunk

  • Members
  • 104 posts
  • Gender:Not Telling
  • Location:Ravens Isle
  • Interests:Other-Life, Pizza, Rocky Horror Picture Show

Posted 12 February 2007 - 10:36 AM

Without any quantitative testing, I'll just say it feels leaps and bounds faster.

#8 Learner

Learner

    Ant

  • Admin
  • 5430 posts
  • Gender:Male
  • Location:MN, USA
  • Interests:Persistent RPG

Posted 12 February 2007 - 02:41 PM

Who other then LabRat has also taken the time to gzip their files? Are you noticing additional speedup with the compressed files also?

Everyone can benefit from the delayed texture loading, but more feedback on the compressed files portion would be good.

#9 Kindar Naar

Kindar Naar

    Beaver

  • Members
  • 98 posts

Posted 12 February 2007 - 05:24 PM

I run the client only with gzipped files.... and after seeing the speed up I'm not going back to regular ones :blush:

#10 Pardu

Pardu

    White Rabbit

  • Members
  • 36 posts

Posted 12 February 2007 - 07:48 PM

compressing files works great!.. I change maps so fast its like being teleported...thank you..and since I usually only post with whining and problems... another .. thank you .. and keep up the great work

#11 Learner

Learner

    Ant

  • Admin
  • 5430 posts
  • Gender:Male
  • Location:MN, USA
  • Interests:Persistent RPG

Posted 13 February 2007 - 12:13 AM

I also checked in support for all .bmp's being able to be gzipped except texturs/cursor.bmp. I don't expect this to affect performance, but it makes it more consistant and can improve OS cache hit rates on system with low memory.

#12 bluap

bluap

    Feran

  • Moderators
  • 2215 posts
  • Gender:Male
  • Location:UK

Posted 13 February 2007 - 12:14 AM

Who other then LabRat has also taken the time to gzip their files? Are you noticing additional speedup with the compressed files also?

I'm only on Linux. I gziped all my .elm and e3d files. I didn't speed test the new code without the gziped files, only "1.3.3 released binary and unziped files" against "latest cvs code and gziped files". I'm assuming I would not otherwise notice the difference of a native build against the released binary.

#13 LabRat

LabRat

    Phantom Warrior

  • Members
  • 2620 posts
  • Gender:Male
  • Location:West Midlands, England, UK
  • Interests:mainly keeping Michelle_F off my ubercool one of a kind *autographed* EL T-shirt

Posted 13 February 2007 - 02:00 AM

If I read correctly, even without gzipping anything you will still see a difference due to stage one - loading only when an item is needed rather than at map load.

Next stage was gzipping the .elm and .e3d files

After that was bitmap compression - I couldn't really see a speed difference but I have no doubt there was one there, however miniscule.

I compressed my exe and dll files with an exe compressor while I was at it (not pertinent to the discussion but I just thought I'd mention it :P)

#14 Learner

Learner

    Ant

  • Admin
  • 5430 posts
  • Gender:Male
  • Location:MN, USA
  • Interests:Persistent RPG

Posted 13 February 2007 - 03:48 PM

If I read correctly, even without gzipping anything you will still see a difference due to stage one - loading only when an item is needed rather than at map load.

Next stage was gzipping the .elm and .e3d files

After that was bitmap compression - I couldn't really see a speed difference but I have no doubt there was one there, however miniscule.

I compressed my exe and dll files with an exe compressor while I was at it (not pertinent to the discussion but I just thought I'd mention it ;))

Correct.

Another side effect of the compression is that in the future auto update and custom looks downloads will be faster as well.




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users