macOS - 64-bit client - help needed

[Ben]

Towards the end of this month Apple will be releasing the next version of macOS, Catalina.

 

One of the more notable un-features of Catalina will be the end of support for 32-bit apps, meaning that the majority of our Mac players will be gradually left out in the cold unless we can come up with a working 64-bit build of the game.

 

Now, I am not a developer, but I have slowly been working through the issues and I think I am close to having a working build. The issue I am stuck on right now is framework-related; I have identified that six of the .frameworks required to build and run the game are not suitable for use with 64-bit:

SDL.framework

SDL_image.framework

SDL_net.framework

vorbis.framework

ogg.framework

cal3d.framework

 

Replacing the old SDL, SDL_image, and SDL_net frameworks with suitable replacements has proven to be easy as these are readily available from libsdl.org. I have also been able to locate some pre-built vorbis and ogg frameworks (built for iOS, but seem to work nonetheless) - though these are of course not ideal.

 

The real stumbling block is the cal3d framework. Despite numerous attempts, I can't figure out how to get it to build correctly.

 

If somebody could provide some expertise (or perhaps even step in and take over) I would be most appreciative, as it would be a great shame for us to lose our Mac community. 

[Ben]

Getting closer:

[10:43:55, /Users/ben/Desktop/Eternal-Lands-master/engine/logging.cpp:310] Log started at: Sun Sep  8 10:43:55 2019 BST
[10:43:55, /Users/ben/Desktop/Eternal-Lands-master/engine/logging.cpp:313] version: 1.9.5
[10:43:55, /Users/ben/Desktop/Eternal-Lands-master/servers.c:118] Error: Fatal error: servers.lst file missing!

Unsure what is causing the servers.lst fatal error, it is definitely present.

[Ben]

Success! I have it running, though there's a bit of weirdness with files needing to be in places they didn't need to be previously.

 

To get it to work I had to manually place servers.lst in ~/Library/Application Support/Eternal Lands/ and edit #data_dir in the el.ini under ~/Library/Application Support/Eternal Lands/main.

[bluap]

14 minutes ago, Ben said:

Success! I have it running, though there's a bit of weirdness with files needing to be in places they didn't need to be previously.

 

To get it to work I had to manually place servers.lst in ~/Library/Application Support/Eternal Lands/ and edit #data_dir in the el.ini under ~/Library/Application Support/Eternal Lands/main.

If I understand what you have said correctly, that condition sounds like the normal way it works to me.  I do not have access to MacOS but normally the game will run if you start it from the directory where the main game data is located, or you have the servers.lst copied and #data_dir set in the el.ini (or the equivalent command line switch).

[Ben]

8 minutes ago, bluap said:

If I understand what you have said correctly, that condition sounds like the normal way it works to me.  I do not have access to MacOS but normally the game will run if you start it from the directory where the main game data is located, or you have the servers.lst copied and #data_dir set in the el.ini (or the equivalent command line switch).

 

Interesting, I must be doing something different to cause it to ignore the servers.lst contained within the .app.

 

The Odie client reads servers.lst from within itself: EternalLands.app/Contents/Resources/data/servers.lst (tested and confirmed). My client seems to ignore this file, even with an identical set of data files 

 

 

[bluap]

So you are not comparing two builds on the latest github code?

It could be something to do with this change:

https://github.com/raduprv/Eternal-Lands/commit/31c03860b7bf6ae60483f722fb03c2f88c0314fc

 

Specifically:

#ifndef WINDOWS
	if (chdir(datadir) != 0)
	{
		LOG_ERROR("%s() chdir(\"%s\") failed: %s\n", __FUNCTION__, datadir, strerror(errno));
	}
#endif //!WINDOWS

Having this, breaks auto restart of the client for non-windows clients.

We should probably change the server.lst loading code to look for the file in datadir if it fails to find it in user space.

Edited September 8, 2019 by bluap
Extra text

[Ben]

I understand why it is behaving differently now 

 

1 hour ago, bluap said:

We should probably change the server.lst loading code to look for the file in datadir if it fails to find it in user space.

 

That would be a great help, without such a change Mac users would have to manually copy servers.lst over to ~/Library/Application Support/Eternal Lands/. 

 

 

[bluap]

I quickly checked and it appears that if datadir is set then the servers.lst file is found there correctly.  This was using the -dir=<path> command line option.  I need to check some more.  Nope, scratch that, I was running from the source code directory which has a servers.lst file! That clean up is a different story.  We need the change I suggested.

Edited September 8, 2019 by bluap

[Ben]

The #data_dir not failing over is causing some other launch problems too, if not manually changed.

 

Odie client behaviour (launches):

[14:26:19, /Users/linke/Desktop/el-devel/elc/macosx/../io/elpathwrapper.c:664] Warning: Didn't find your data_dir, using the current directory instead. Please correct this in your el.ini . Given data_dir was: "c:\Program Files\Eternal Lands\/". Using "/Applications/EternalLands.app/Contents/Resources/data".

 

 

New client behaviour (crashes):

[14:28:09, /Users/ben/Desktop/Eternal-Lands-master/io/elpathwrapper.c:664] Warning: Didn't find your data_dir, using the current directory instead. Please correct this in your el.ini . Given data_dir was: "c:\Program Files\Eternal Lands\/". Using "/".

[14:28:09, /Users/ben/Desktop/Eternal-Lands-master/init.c:736] Error: Fatal error while loading data files. Either set the data_dir correctly or run from the data directory.

 

I'm guessing this wouldn't be a problem for new installs (it could be added to the el.ini in the included data files, right?), but it would cause problems for people upgrading.

Edited September 8, 2019 by Ben

[bluap]

I've tried reverting the above change and there are still issues.  If you start the client in the datadir, it all works, that's how it works for windows.   For the Linux builds I do, I copy the needed files for new installs.  I have to go AFK for a while but will have another look when I get back.

Edited September 8, 2019 by bluap

[Ben]

 

12 minutes ago, bluap said:

I've tried reverting the above change there are still issues.  If you start the client in the datadir, it all works, that's how it works for windows.   For the Linux builds I do, I copy the needed files for new installs.  I have to go AFK for a while but will have another look when I get back.

 

Things work a little differently on macOS, unfortunately. Nothing is really installed anywhere as almost everything (besides the user data) is self-contained within the EternalLands.app in /Applications. You can't really start the game from the datadir because the datadir and executable must be in a prescribed place within the .app - there isn't much flexibility.

 

12 minutes ago, bluap said:

I have to go AFK for a while but will have another look when I get back.

 

Thank you for taking a look at this, Bluap, I really appreciate it. Enjoy the rest of the weekend. 

Edited September 8, 2019 by Ben

[bluap]

OK.  I've checked some more and I'm pretty sure the change I pointed you to is not your issue. I was pretty sure when I made it that it was not going to have additional impact other than fixing the restart issue and I've just tried a whole bunch of testing, again, to make myself pretty sure that's the case.  I can only test on Linux though.

 

However, if you are using the Sir Odie build from 06.05.2019, and that worked as you'd expect, then I'm doubly sure the above change is not your issue.  That build was done after the above change was committed on 18th April.

 

As far as I can tell.  You either have to 1) start the client with the data directory as your working directory, 2) have a copy of servers.lst in your working directory, or 3) have a copy of servers.lst in the root directory of the users configuration directory tree.

 

I presume Sir Odie does not modify the client in some way to make his builds work on MacOS.

[Ben]

On 08/09/2019 at 7:16 PM, bluap said:

However, if you are using the Sir Odie build from 06.05.2019, and that worked as you'd expect, then I'm doubly sure the above change is not your issue.  That build was done after the above change was committed on 18th April.

I have just re-downloaded the latest Sir_Odie build to double-double check, and it does indeed work as expected. Weird.

 

On 08/09/2019 at 7:16 PM, bluap said:

As far as I can tell.  You either have to 1) start the client with the data directory as your working directory, 2) have a copy of servers.lst in your working directory, or 3) have a copy of servers.lst in the root directory of the users configuration directory tree.

Could we perhaps have the client automatically copy the included servers.lst to the user configuration folder, the same way it builds the rest of the files in there if it can't find them? If we can't figure out why my build no longer utilises the internal servers.lst file, that would save an additional step for Mac users installing the client.

 

I'll try and reach out to Sir_Odie and see whether he does indeed use some trick to make his build behave differently. Although I am using the same Xcode project, it could just be down to the newer Xcode version I am using (he builds his client on an older machine).

 

Edit: Just built the source as it was on the 17th of April (before the change you mentioned) to double-double-triple check, still seeing the strange different behaviour.

Edited September 11, 2019 by Ben

[Ben]

I have heard back from Sir_Odie, his help pinpointing the issue proved invaluable. It turns out there was a small issue in the new SDL_main.m file required by the upgraded version of the SDL.framework (1.2) I used for 64-bit compilation.

 

Now we have it working fine under 10.14, I have upgraded one of my machines to 10.15 (Catalina) for testing and I'm seeing a strange graphical bug which is not present on 10.14. Have you ever come across that before? The EL log isn't giving me any clues as to what might be the issue, displaying only the same information as both the current Odie and 64-bit builds do on a 10.14 machine.

Edited September 11, 2019 by Ben

[Ben]

Resolution issue is fixed, thanks to some genius-level thinking from Revi.

 

It turns out there was a slight change in the way 10.15 handles lower resolution apps and this needed to be specified explicitly in the info.plist file:

NSHighResolutionCapable = NO

Pending more testing, we provisionally have a working 64-bit client compatible with the upcoming version of macOS 

[bluap]

@Ben nice work! And, no I'd not seen a graphical bug like that before.

[vinoveritas]

Just my 2 cent should  it not help to put  in the init.c those 2 lines  after the configdir

#ifdef OSX
char configdir[256]="../Resources/data";

char datadir[256]="../Resources/data";

#else

char configdir=."./";

...

#endif

#endif

to make sure that if OSX is used it checks for the Datadir inside the app first?

 

I was trying to get any way i could think of to compile and run a mac client without a mac but all of those efforts did fail in the end so i went to this.

 

Edited February 24, 2020 by vinoveritas