New Encyclopedia Xml Data Format

[Lachesis]

Dear encyclopedia volunteers,

 

as you know, the current encyclopedia data format is not comfortable at all to write, maintain and translate. So I decided to implement a new format based on semantic markup. If you like to see the XML subset definitions, you can find them at the Encyclopedia CVS repository in the /dtd directory. For a quick guide on how to use the format, I also created an example file that explains the most important concerns. Of course, it is not normative, for reference please check the XML Schema files or ask. If you encounter odds or weirdnesses in the format, please don't hesitate to tell since it is not finished yet. Once the version bumps to 1.0, I will start writing code depending on the format, and changes will be more difficult.

 

With regards

Lachesis

 

Edit: Inserted links, deleted the quote of the example file.

Edit: Please also see the post about the visual format

Edited November 27, 2005 by Lachesis

[Wytter]

Looks good. I have 1 comment though on the following:

 

<creature>
 <name>Fluffy Rabbit</name>

 <description>Goes here ... </description>

 <hostile/>
 <!-- only for summonable creatures: -->
 <ethereal-cost>?</ethereal-cost>
 <ingredients>
  <amount>1</amount><name>White Rabbit Fur</name>
  <amount>1</amount><name>Bone</name>
  <amount>1</amount><name>Meat</name>
  <amount>1</amount><name>Enriched Life Essence</name>
 </ingredients>
 <required nexus="animal">4</required>
</creature>

I think it'd be cleaner if it was:

<creature>
 <name>Fluffy Rabbit</name>

 <description>Goes here ... </description>

 <hostile/>
 <!-- only for summonable creatures: -->
 <ethereal-cost>?</ethereal-cost>
 <ingredients>
  <ingredient amount="1">White Rabbit Fur</ingredient>
  <ingredient amount="1">Bone</ingredient>
  <ingredient amount="1">Meat</ingredient>
  <ingredient amount="1">Enriched Life Essence</ingredient>
 </ingredients>
 <required nexus="animal">4</required>
</creature>

That goes for all of the <amount></amount><name></name> tags...

Edited April 16, 2005 by Wytter

[crusadingknight]

I can't wait for client support . (As well, I like to see the output of my work).

[Lachesis]

  <ingredients>
  <ingredient amount="1">White Rabbit Fur</ingredient>
  <ingredient amount="1">Bone</ingredient>
  <ingredient amount="1">Meat</ingredient>
  <ingredient amount="1">Enriched Life Essence</ingredient>
 </ingredients>

153569[/snapback]

Yes, I thought about that, Malaclypse also suggested this. Since you both prefer it and since the amount is always just a number, I think I'll change it. However, if we do it this way, the ingredients tag is superfluous. I think I'll also make the amount attribute optional and give it a default value of 1 for convenience.

 

With regards

Lachesis

 

P.S. Changes done, updated files are in CVS. In addtion, I added the missing example for the spell topic to the example file. Also added a conversion script to CVS that can convert files written conforming earlier versions of semantic.xsd (from 0.9.1 on) to the current format. I'll try to keep this script up to date with changes in semantic.xsd. Unfortunately, the script uses XSLT 2.0 to realise an incremental update, so you will not be able to use it with most XSLT processors. I'm using Saxon, that is free in its non-schema-aware version.

P.P.S Sorry, Crusading Knight, but a lot of things need to be done before the client code can be written. For instance, the visual markup format needs to be completed. This format is what the client will parse when displaying the encyclopedia (or later on, maybe any book). The XSLT script doing the transformation from semantic to visual format isn't even started yet.

[Lachesis]

Lazy so far today, but I figure time may be a little on my side... anyway, I got capes and cloaks done, and added an example creature and perk.

 

--BTW, can we possibly add an "opposing perk" element (just a string) to the perk element? I think it wouldn't look good in the description element.

154162[/snapback]

(Took this into this thread in order to keep discussion about the format in one place) You mean for perks that cannot be used at the same time?

[crusadingknight]

Yes.

[Lachesis]

Done. Kept the version number since the change does still validate all documents it did before.

[Lachesis]

Templates for semantic markup

 

Upon transformation, the XSLT script has to insert some language-specific entities such as labels for properties like "weight", "recommended level" and so on. These are retrieved from the source file using templates, so that the translator does not need to translate 'alchemy' n+1 times in his document. I added the hooks for inserting such templates to the semantic format, however their format itself is specified in a separate schema and namespace. I also added templates to the example file for semantic markup. Please have a look at them and send your comments.

 

Visual markup

 

I have created an example file for the visual markup* that explains the intended use. I know that the first transformation script from semantic to visual markup has been written by myself, but I'm planning to delegate its maintenance to any volunteer that speaks XSLT 2.0 a.s.a.p. So although there is probably no one for that task yet, I need comments on the visual format, as I'm going to start its implementation in C now. Once it's done, changes will be harder.

 

With regards

Lachesis

 

*) Please don't edit that file directly. Instead, modify its source file in semantic markup and repeat the transformation.

[Lachesis]

Hint: Since there were no comments, I'll implement this now.

 

With regards

Lachesis

[jakoblin]

so when will it be out ?

[i92guboj]

Hi all and thanks to CrusadingKNIGHT for pointing me here for this question.

 

The question: so, if I want to make a translation for the encyclopedia and help pages, can I start by modifying the files from here: http://www.eternal-lands.com/forum/index.php?showtopic=13861 ????

 

I choosed the zip version since is easier for me to download (yes, I still have a 56k modem ). I have time, so if I can help count on me.

 

Regards.

[Lachesis]

so when will it be out ?

162075[/snapback]

I don't know, looking at my spare time in the near future, I estimate it'll take still at least three months until the source will be mature enough to go into the ordinary CVS client.

 

if I want to make a translation for the encyclopedia and help pages, can I start by modifying the files from here: http://www.eternal-lands.com/forum/index.php?showtopic=13861 ????

Yes, you can do that. However, be aware that you'll need to redo this work when the new data format will go into effect. CK currently is translating the English version to the new format, it would be great if you could help him with this task as well. Above you can find links to example files that explain the new format in detail, the old format is described by CK in a pinned thread IIRC.

 

With regards

Lachesis

[crusadingknight]

I don't know, looking at my spare time in the near future, I estimate it'll take still at least three months until the source will be mature enough to go into the ordinary CVS client.

163507[/snapback]

 

Hrm, just to double-check - no chance of finishing that in 3 weeks, is there? I just wanted to eliminate that remote possibility, because, if not, I have to start backporting stuff to the old format. As well, I'll have to recruit a few maintainers, to keep things up-to-date during testing (which I _suspect_ we'll do).

[Lachesis]

I'm very sorry, but I don't see any chance to get it done that fast. I have about 90-95% of the parsing code done, and it's already 1,500 lines of code. I'm currently not working on it because there are some other things that I have to do first. I can't estimate at all how much work it will be to write the typesetting/composition code, but it'll probably take significantly longer. The conversion of the new format back to the old one however should be possible to do automatically using XSLT, at least the biggest part of it. This should both apply to encyclopedia/help and book files.

 

With Regards

Lachesis

[crusadingknight]

OK, don't worry, I was just checking; Now I know what to add to my ToDo list.

[crusadingknight]

Meep/bump... I take it this initiative is semi-dead?

[Lachesis]

No, but it's taking a long time.

[Lachesis]

Unfortunately, all uncommitted work has been lost due to a hardware failure. I'm currently working on another project so I can't continue this atm, if someone wants to take it up, all documentation on the formats can be found on freepository and is linked in this thread.

 

The parser part of the implementation is almost done, it does the following things:

• building an internal representation of the XML tree
  
• propagating attribute values according to inheritance rules
  
• decoding and interlinking hypertext labels and references
  
• (missing) destroying an internal tree representation
  

The code should be pretty self-explanatory.

 

Sorry for the inconvenience,

Lachesis