ZLine Extension: Bandwidth usage reduction
 

News:

29 December 2022 - PtokaX 0.5.3.0 (20th anniversary edition) released...
11 April 2017 - PtokaX 0.5.2.2 released...
8 April 2015 Anti child and anti pedo pr0n scripts are not allowed anymore on this board!
28 September 2015 - PtokaX 0.5.2.1 for Windows 10 IoT released...
3 September 2015 - PtokaX 0.5.2.1 released...
16 August 2015 - PtokaX 0.5.2.0 released...
1 August 2015 - Crowdfunding for ADC protocol support in PtokaX ended. Clearly nobody want ADC support...
30 June 2015 - PtokaX 0.5.1.0 released...
30 April 2015 Crowdfunding for ADC protocol support in PtokaX
26 April 2015 New support hub!
20 February 2015 - PtokaX 0.5.0.3 released...
13 April 2014 - PtokaX 0.5.0.2 released...
23 March 2014 - PtokaX testing version 0.5.0.1 build 454 is available.
04 March 2014 - PtokaX.org sites were temporary down because of DDOS attacks and issues with hosting service provider.

Main Menu

ZLine Extension: Bandwidth usage reduction

Started by Jove, 31 October, 2005, 20:06:09

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

Jove

Hi,

        i would like to (officially) request inclusion of the ZLine extention into PtokaX. Depending on your hubsoft architecture, this will allow bandwidth reductions of upto 50% per user.
        For more information, please check out the DC++ Wiki.  For a DC++ patch, check out the DC++ bugzilla.

A list of clients and hubsofts supporting this extension can be be found on the DC++ Wiki.

thank you

bastya_elvtars

#1
This would affect data-sending funbctions in the API (at least the instructions they trigger in C), but looks nice.
Everything could have been anything else and it would have just as much meaning.

Jove

Thank you.

I've been gathering support from various hubs and clients during the past week or so.  They are added to the Wiki entry when reported.

I would really like to see this go in. To be honest, it might require a bit more work in the hubsoft to really take advantage of the feature though. The bandwidth saving rises with the size of the compressed buffer. It does miracles for things like userlists....

bastya_elvtars

I would love to see this feature. Even my 100Mbit connection is stressed when I connect to 10000+ hubs. :D
Everything could have been anything else and it would have just as much meaning.

Pothead

.
#4
QuoteOriginally posted by bastya_elvtars
I would love to see this feature. Even my 100Mbit connection is stressed when I connect to 10000+ hubs. :D
While you might see it as a way of saving bandwidth with your leeching  8o (hehe, j/k), i see it as larger hubs.  ;)

Pothead

Just to let you know status of this.  Arnetheduck rejected ZLine in it's current form, and suggested an alternative way of doing it.  Jove has constructed another patch to do it this way, and this has been accepted into the DC++ cvs / svn. :)

http://dcpp.net/bugzilla/show_bug.cgi?id=834 For the patch.  For any questions about how it works, just ask me (i gotta vague idea), or ask Jove (he knows much more, lol). :)

bastya_elvtars

OK, so we can expect this in next DC++ release, and the support in PtokaX is being worked on, AFAIK.  ;)
Everything could have been anything else and it would have just as much meaning.

Pothead

Quote from: bastya_elvtars on 27 February, 2006, 17:37:20
OK, so we can expect this in next DC++ release, and the support in PtokaX is being worked on, AFAIK.  ;)
Yes, unfortuantelly it's (ZPipe) is different to how ZLine works, so clients / hubs which have already implemented ZLine will have to change it. :P

bastya_elvtars

Yeah, arnetheduck is very namby-pamby.  ::)
If he has hiccups, it's because latest DC++ crashed my PC 3 times in 8 hours, and now he buggers with such nonimportant stuff. Thumbs up? down? what?
Everything could have been anything else and it would have just as much meaning.

PPK

From patch it looks like ZPipe means to compress all data :o I am not sure if this is possible to do without high cpu usage  :(
"Most of you are familiar with the virtues of a programmer. There are three, of course: laziness, impatience, and hubris." - Larry Wall

Pothead

From what i understood, from questioning Jove on this, hub sends $ZOn| or sommit, and it opens up a compressed stream ?.  Once data finished being sending down it, the stream is closed, and the client reverts to back to normal mode.  If more compressed data is to be sent, hub sends $ZOn| again first to open up another compressed stream.
Does that make sense (i have no idea about sockets / streams and stuff). ?

Jove

ZPipe does indeed compress the data. It does not necessarily use a lot of cpu. It all depends on how your hub is structured. Aquila creates generic buffers send regularly to all users (with searches for example) those buffers are compressed once and then send to all users. The cpu usage of the compression is negligable. Just compressing everything for each users individually is indeed prohibitive.

This is why ZPipe is structured as it is. You turn it on with $ZOn| and as soon as the compressed stream ends, you fall back to normal mode.

PPK

Ok, now i understand how it is works ;) And is easy to me to change from Zline to ZPipe  8)
"Most of you are familiar with the virtues of a programmer. There are three, of course: laziness, impatience, and hubris." - Larry Wall

Pothead

Quote from: PPK on 28 February, 2006, 03:18:32
Ok, now i understand how it is works ;) And is easy to me to change from Zline to ZPipe  8)
Nice.  Just a little side note . . . . DC++ sends
ZPipe0
in it's supports.  Which i think is only change to that patch. :)

bluebear

I think no matter what this will be a cpu hog on hubs with a large amount of users.. 5000 users sends a lot of searches; means that px will have to compress data often. However this will indeed be a good feature for a small hub on a small connection. Personally i think that making a new protocol would be much better if you want to save bandwidth and cpu. So instead of a string based protocol it should be a binary protocol. That could offer upto 40-50% bandwidth saves and even more on the cpu.

Example copy paste into notpad or a program that show all chars in same size:

The current NMDC proto:

    1? ? ? ? ?2? ? ? ? ?3? ? ? ? ?4? ? ? ? ?5? ? ? ? ?6? ? ? ? ?7? ? ? ? ?8? ? ? ? ?9? ? ? ? ?10
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
-----------------------------------------------------------------------------------------------------
$ConnectToMe nickname 127.0.0.1:1024|? ? ? ? ? ?<- bandwith used 37 bytes


A new binary proto:

    1? ? ? ? ?2? ? ? ? ?3? ? ? ? ?4? ? ? ? ?5? ? ? ? ?6? ? ? ? ?7? ? ? ? ?8? ? ? ? ?9? ? ? ? ?10
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
-----------------------------------------------------------------------------------------------------
1nickname5IPPORT0? ? ? ? ? ? ? <- bandwidth used 17 bytes
0

char 10 = $ConnecToMe (takes 1 byte instead of 12 bytes)
char 5? = delimiter
char 0? = end of message (like "|")
IP = unsigned long (takes 4 bytes instead of upto 15 bytes)
PORT = unsigned short (takes 2 bytes instead of upto 4 bytes)

You must also remember that the hub must to compare the clients
data against strings like "$ConnectToMe" it takes much much more
resources then just checking the value of a single byte.
Sincerely,
bluebear
--
http://www.thewildplace.dk/ is is closed - Use the following mirrors instead
http://bluebear.psycho-chihuahua.net
http://pxextension.piratez.dk/
[Lua extensions - Chat stats - YnHub PMSpy - DC Source code - and more]

Troubadour

Can this also be used for the webserver?

Regards,

Troubadour

** Guardian Forum **

hubaddy:   nederfun.no-ip.com

PPK

ZPipe in PtokaX working... but it looks like DC++ 0.687 have bug in support and sometimes show parts of unzlibed data in chat ::)
"Most of you are familiar with the virtues of a programmer. There are three, of course: laziness, impatience, and hubris." - Larry Wall

Jove

Yes. Problem is fixed and patch send.

Btw, the ZPipe0 is a temporary support since ZPipe is still in testphase.

PPK

Ok, patch aplied and now it works without problems :)
"Most of you are familiar with the virtues of a programmer. There are three, of course: laziness, impatience, and hubris." - Larry Wall

Troubadour

Regards,

Troubadour

** Guardian Forum **

hubaddy:   nederfun.no-ip.com

SMF spam blocked by CleanTalk