PtokaX forum

PtokaX => Feature Proposals => Topic started by: PPK on 06 August, 2011, 23:09:35

Poll
Question: Should xml files be replaced by text/binary files?
Option 1: Yes! votes: 8
Option 2: NO! votes: 3
Title: Replacing xml configuration files with text/binary files
Post by: PPK on 06 August, 2011, 23:09:35
One of PtokaX problems is storing data (registered users, banlist etc.) in xml files. Reading and writing large xml files is slow and can't be much improved. Another problem with xml is dependency on tinyxml, thing that make compilation on unix systems more complicated that it should be.

So i'm here with idea to replace slow xml with better format for data saving.
Idea is to use simple binary files that i'm designed (and i will use it to store GUI settings) to store PtokaX configuration files. Not only that this should be faster, but it will allow to save changes in regs/bans without writting whole file as it is needed in xml.
In case when it will be needed then files like settings can be saved as simple text files, to allow users to change them...
Of course for few versions PtokaX will be able to read old xml files and convert them to new data format.

So please vote above if this change should be done, and post below if you want to change all files to binary or some files can be saved as plain-text  8)
Title: Re: Replacing xml configuration files with text/binary files
Post by: bastya_elvtars on 09 August, 2011, 18:14:09
How about language files?
Oh, and why not use Lua for editable configuration, so users editing the PtokaX config files will have a familiar syntax when configuring scripts?
Title: Re: Replacing xml configuration files with text/binary files
Post by: PPK on 09 August, 2011, 18:28:11
Quote from: bastya_elvtars on 09 August, 2011, 18:14:09
why not use Lua for editable configuration
Because that not improve performance...
Title: Re: Replacing xml configuration files with text/binary files
Post by: bastya_elvtars on 09 August, 2011, 20:47:37
Quote from: PPK on 09 August, 2011, 18:28:11
Because that not improve performance...

I know but for language files and the editable configs the performance penalty shouldn't be an issue as they aren't read/written all the time. I am not talking of huge lists (regs, bans) here.
Title: Re: Replacing xml configuration files with text/binary files
Post by: pHaTTy on 09 August, 2011, 21:07:14
Since the beginning of XML's existence, I've always said that XML is slow, so i agree that you should put together a binary format for bans and registered users, as for language files, as bastya pointed out, they are not re-loaded repeatedly and therefore the performance decrease would be unnoticeable, especially for todays specifications of CPU's, which would allow for easier editing of languages for third-party translators.

-pHaTTy
Title: Re: Replacing xml configuration files with text/binary files
Post by: bastya_elvtars on 09 August, 2011, 22:06:07
Moreover, settings could directly be read from scripts and there would be no need to maintain the C code of PtokaX wrt the retrieval of them. But the strongest argument IMHO is that since many scripts have configuration options (obviously) written in Lua, PtokaX should also have them this way, so everything could be set up the same way. Even there could be a standardized configuration format, so configuring a script would provide the same 'experience' as configuring PtokaX. It would be a dream for the console version users.
(Personal opinion: XML is not good for humans nor computers :D)
Title: Re: Replacing xml configuration files with text/binary files
Post by: WAJIM on 10 August, 2011, 10:29:40
I've no problems with xml's load/save speed. My hub has ~5000 regnicks.  ::)

I vote against a binary format since it will be very difficult for editing manually.  >:(
Title: Re: Replacing xml configuration files with text/binary files
Post by: PPK on 21 August, 2011, 21:46:36
Quote from: WAJIM on 10 August, 2011, 10:29:40
I've no problems with xml's load/save speed. My hub has ~5000 regnicks.  ::)
If you have fast CPU then you don't notice that. But hub is freezed for few hundred miliseconds on reglist save  :-\

Test on my AMD Athlon X2 5050e (2.6GHz) with 10000 regs.
XML file: Load 312ms. Save 297ms. File size 1 477 892 bytes.
Binary file: Load 16ms. Save 0ms (crazy :o ). File size 387 827 bytes.


Quote from: WAJIM on 10 August, 2011, 10:29:40
I vote against a binary format since it will be very difficult for editing manually.  >:(
You can add/change/remove regs from GUI or from Lua, i'm sure that 99,9% of users don't need to edit regs/bans file manually  ::)
Title: Re: Replacing xml configuration files with text/binary files
Post by: WAJIM on 24 August, 2011, 18:07:58
Quote from: PPK on 21 August, 2011, 21:46:36
If you have fast CPU then you don't notice that. But hub is freezed for few hundred miliseconds on reglist save  :-\
Even 300ms freezes every 5 minutes cfg autosave (for my scripts) - not critical.  ::)
Title: Re: Replacing xml configuration files with text/binary files
Post by: PPK on 24 August, 2011, 18:44:45
You like laggy hub, others don't. They complained that hub lags when PtokaX saving data, so i'm removed automatic save after changes. Now they complains that they losing data because hub is not saving data after change... I can't fix both when xml is used for data  ::)
Title: Re: Replacing xml configuration files with text/binary files
Post by: pHaTTy on 25 August, 2011, 11:02:50
why not save the data on a separate thread.... it should be easy to implement, setting the thread to low priority will allow auto saves and not cause any lag on the hub, in the time that it saves set a var so when the hub is saving any new data will be added to a data queue, and then added to the list for saving next time around.

-pHaTTy
Title: Re: Replacing xml configuration files with text/binary files
Post by: PPK on 25 August, 2011, 15:17:22
Saving in thread will be bad for hub performance, because it is gives more work for main thread ::)
Title: Re: Replacing xml configuration files with text/binary files
Post by: pHaTTy on 26 August, 2011, 14:52:36
that is absolute rubbish.
Title: Re: Replacing xml configuration files with text/binary files
Post by: PPK on 22 October, 2011, 20:32:59
Voting locked. Reglist and banlist will be in binary files. Human readable backups (xml/txt/anything else) can be created from Lua API  :P