@NutsAboutAmiga

Quote:

Way do you use TCP? Way not use UDP, do you need all packages to get received? If you drop one Mouse coordinates, it should not big deal if you get on the next package.

TCP is alot easier And I wasn't expecting apacket drop issue over the LAN

But note whilst mouse movements might be discardable, the other events mouse button up / down , keyboard up down , mouse wheel etc et are more critical. Also the control commands need to be reliable, else you could end up with both srever and client without a mouse or both going at the same time.

AS far as I know synergy uses TCP

_________________
BroadBlues On Blues BroadBlues On Amiga Walker Broad

@broadblues

I did some testing this afternoon to assess how many drops I was getting in each direction.

Using tcpdump redirected to a PIPE: and parseing the output with a rexx script to pick out packets with duplicate ack and identical MKShare packet type.

Loging on X1000:

10.Beta:> tcpdump -X port 35402 >PIPE:logit
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on RTL8139, link-type EN10MB (Ethernet), capture size 68 bytes
39121 packets captured
39886 packets received by filter
0 packets dropped by kernel

13.Beta:> rx ram:drop.rx 
14:31:35.834013 IP X1000.1024 > SAM.35402: P 22483:22488(5) ack 129053 win 65535
0x0000	 4500 002d 645d 0000 4006 9254 c0a8 0165	E..-d]..@..T...e
0x0010	 c0a8 0164 0400 8a4a 16a8 3306 bda9 22a5	...d...J..3...".
0x0020	 5018 ffff cade 0000 5243 5644 00       	P.......RCVD.
14:35:45.675694 IP X1000.1024 > SAM.35402: P 36988:36993(5) ack 212718 win 65535
0x0000	 4500 002d 713c 0000 4006 8575 c0a8 0165	E..-q rx ram:drop.rx 
Suspected packet drop:
14:51:07.447140 IP X1000.1024 > SAM.35402: P 19470:19475(5) ack 113706 win 65535
0x0000	 4500 002d 8ae9 0000 4006 6bc8 c0a8 0165	E..-....@.k....e
0x0010	 c0a8 0164 0400 8a4a 16a8 d8c9 bdac cad1	...d...J........
0x0020	 5018 ffff 7ceb 0000 5243 5644 00       	P...|...RCVD.

14:51:08.620154 IP X1000.1024 > SAM.35402: P 19470:19475(5) ack 113706 win 65535
0x0000	 4500 002d 8aec 0000 4006 6bc5 c0a8 0165	E..-....@.k....e
0x0010	 c0a8 0164 0400 8a4a 16a8 d8c9 bdac cad1	...d...J........
0x0020	 5018 ffff 7ceb 0000 5243 5644 00       	P...|...RCVD.
Suspected packet drop:
14:57:22.451578 IP X1000.1024 > SAM.35402: P 49234:49239(5) ack 285350 win 65535
0x0000	 4500 002d a4e6 0000 4006 51cb c0a8 0165	E..-....@.Q....e
0x0010	 c0a8 0164 0400 8a4a 16a9 4d0d bdaf 694d	...d...J..M...iM
0x0020	 5018 ffff 6a28 0000 5243 5644 00       	P...j(..RCVD.

14:57:23.537032 IP X1000.1024 > SAM.35402: P 49234:49239(5) ack 285350 win 65535
0x0000	 4500 002d a4e9 0000 4006 51c8 c0a8 0165	E..-....@.Q....e
0x0010	 c0a8 0164 0400 8a4a 16a9 4d0d bdaf 694d	...d...J..M...iM
0x0020	 5018 ffff 6a28 0000 5243 5644 00       	P...j(..RCVD.
Suspected packet drop:
15:01:41.888841 IP X1000.1024 > SAM.35402: P 69097:69102(5) ack 400265 win 65535
0x0000	 4500 002d b65a 0000 4006 4057 c0a8 0165	E..-.Z..@.@W...e
0x0010	 c0a8 0164 0400 8a4a 16a9 9aa4 bdb1 2a30	...d...J......*0
0x0020	 5018 ffff 5bac 0000 5243 5644 00       	P...[...RCVD.

15:01:42.901077 IP X1000.1024 > SAM.35402: P 69097:69102(5) ack 400265 win 65535
0x0000	 4500 002d b65b 0000 4006 4056 c0a8 0165	E..-.[..@.@V...e
0x0010	 c0a8 0164 0400 8a4a 16a9 9aa4 bdb1 2a30	...d...J......*0
0x0020	 5018 ffff 5bac 0000 5243 5644 00       	P...[...RCVD.

8.AmigaOS4:Documentation/Roadshow/TCP-Dump> tcpdump >PIPE:logit -X port 35402
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on PPC440EP_ETH, link-type EN10MB (Ethernet), capture size 68 bytes
39107 packets captured
40115 packets received by filter
0 packets dropped by kernel

10.AmigaOS4:Documentation/Roadshow/TCP-Dump> rx drop.rx 
Suspected packet drop:
14:48:27.181571 IP amiga.35402 > X1000.1024: P 67555:67584(29) ack 11411 win 65535
0x0000	 4500 0045 7119 0000 4006 8580 c0a8 0164	E..Eq...@......d
0x0010	 c0a8 0165 8a4a 0400 bdac 18d8 16a8 ba29	...e.J.........)
0x0020	 5018 ffff fbd4 0000 5241 574d 4f55 5345	P.......RAWMOUSE

14:48:28.229531 IP amiga.35402 > X1000.1024: P 67555:67584(29) ack 11411 win 65535
0x0000	 4500 0045 711a 0000 4006 857f c0a8 0164	E..Eq...@......d
0x0010	 c0a8 0165 8a4a 0400 bdac 18d8 16a8 ba29	...e.J.........)
0x0020	 5018 ffff fbd4 0000 5241 574d 4f55 5345	P.......RAWMOUSE
Suspected packet drop:
14:51:05.977106 IP amiga.35402 > X1000.1024: P 113116:113146(30) ack 19256 win 65535
0x0000	 4500 0046 77de 0000 4006 7eba c0a8 0164	E..Fw...@.~....d
0x0010	 c0a8 0165 8a4a 0400 bdac cad1 16a8 d8ce	...e.J..........
0x0020	 5018 ffff dc47 0000 5241 574d 4f55 5345	P....G..RAWMOUSE

14:51:07.287246 IP amiga.35402 > X1000.1024: P 113116:113146(30) ack 19256 win 65535
0x0000	 4500 0046 77e1 0000 4006 7eb7 c0a8 0164	E..Fw...@.~....d
0x0010	 c0a8 0165 8a4a 0400 bdac cad1 16a8 d8ce	...e.J..........
0x0020	 5018 ffff dc47 0000 5241 574d 4f55 5345	P....G..RAWMOUSE
Suspected packet drop:
14:53:30.091737 IP amiga.35402 > X1000.1024: P 149143:149172(29) ack 25531 win 65535
0x0000	 4500 0045 7d41 0000 4006 7958 c0a8 0164	E..E}A..@.yX...d
0x0010	 c0a8 0165 8a4a 0400 bdad 578c 16a8 f151	...e.J....W....Q
0x0020	 5018 ffff 86f1 0000 5241 574d 4f55 5345	P.......RAWMOUSE

14:53:31.329298 IP amiga.35402 > X1000.1024: P 149143:149172(29) ack 25531 win 65535
0x0000	 4500 0045 7d42 0000 4006 7957 c0a8 0164	E..E}B..@.yW...d
0x0010	 c0a8 0165 8a4a 0400 bdad 578c 16a8 f151	...e.J....W....Q
0x0020	 5018 ffff 86f1 0000 5241 574d 4f55 5345	P.......RAWMOUSE
Suspected packet drop:
14:59:09.430784 IP amiga.35402 > X1000.1024: P 395110:395139(29) ack 67750 win 65535
0x0000	 4500 0045 a019 0000 4006 5680 c0a8 0164	E..E....@.V....d
0x0010	 c0a8 0165 8a4a 0400 bdb1 185b 16a9 963c	...e.J.....[...<
0x0020	 5018 ffff 223a 0000 5241 574d 4f55 5345	P...":..RAWMOUSE

14:59:10.500681 IP amiga.35402 > X1000.1024: P 395110:395139(29) ack 67750 win 65535
0x0000	 4500 0045 a01a 0000 4006 567f c0a8 0164	E..E....@.V....d
0x0010	 c0a8 0165 8a4a 0400 bdb1 185b 16a9 963c	...e.J.....[...<
0x0020	 5018 ffff 223a 0000 5241 574d 4f55 5345	P...":..RAWMOUSE
Suspected packet drop:
15:01:39.846283 IP amiga.35402 > X1000.1024: P 399675:399706(31) ack 68883 win 65535
0x0000	 4500 0047 a154 0000 4006 5543 c0a8 0164	E..G.T..@.UC...d
0x0010	 c0a8 0165 8a4a 0400 bdb1 2a30 16a9 9aa9	...e.J....*0....
0x0020	 5018 ffff d0c7 0000 5241 574d 4f55 5345	P.......RAWMOUSE

15:01:41.229140 IP amiga.35402 > X1000.1024: P 399675:399706(31) ack 68883 win 65535
0x0000	 4500 0047 a156 0000 4006 5541 c0a8 0164	E..G.V..@.UA...d
0x0010	 c0a8 0165 8a4a 0400 bdb1 2a30 16a9 9aa9	...e.J....*0....
0x0020	 5018 ffff d0c7 0000 5241 574d 4f55 5345	P.......RAWMOUSE
Suspected packet drop:
15:05:07.754563 IP amiga.35402 > X1000.1024: P 459478:459507(29) ack 79616 win 65535
0x0000	 4500 0045 ab08 0000 4006 4b91 c0a8 0164	E..E....@.K....d
0x0010	 c0a8 0165 8a4a 0400 bdb2 13cb 16a9 c496	...e.J..........
0x0020	 5018 ffff f860 0000 5241 574d 4f55 5345	P....`..RAWMOUSE

15:05:08.797707 IP amiga.35402 > X1000.1024: P 459478:459507(29) ack 79616 win 65535
0x0000	 4500 0045 ab09 0000 4006 4b90 c0a8 0164	E..E....@.K....d
0x0010	 c0a8 0165 8a4a 0400 bdb2 13cb 16a9 c496	...e.J..........
0x0020	 5018 ffff f860 0000 5241 574d 4f55 5345	P....`..RAWMOUSE
Suspected packet drop:
15:06:48.877454 IP amiga.35402 > X1000.1024: P 517634:517664(30) ack 89531 win 65535
0x0000	 4500 0046 b34c 0000 4006 434c c0a8 0164	E..F.L..@.CL...d
0x0010	 c0a8 0165 8a4a 0400 bdb2 f6f7 16a9 eb51	...e.J.........Q
0x0020	 5018 ffff 94a6 0000 5241 574d 4f55 5345	P.......RAWMOUSE

15:06:50.327512 IP amiga.35402 > X1000.1024: P 517634:517664(30) ack 89531 win 65535
0x0000	 4500 0046 b34d 0000 4006 434b c0a8 0164	E..F.M..@.CK...d
0x0010	 c0a8 0165 8a4a 0400 bdb2 f6f7 16a9 eb51	...e.J.........Q
0x0020	 5018 ffff 94a6 0000 5241 574d 4f55 5345	P.......RAWMOUSE

_________________
BroadBlues On Blues BroadBlues On Amiga Walker Broad

@broadblues

So some reverse logs:

Logging on X1000:

10.Beta:> tcpdump -X port 35402 >PIPE:logit
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on RTL8139, link-type EN10MB (Ethernet), capture size 68 bytes
62851 packets captured
63502 packets received by filter
0 packets dropped by kernel

13.Beta:> rx ram:drop.rx 
Suspected packet drop:
15:33:05.628817 IP X1000.35402 > SAM.1135: P 163800:163829(29) ack 28756 win 65535
0x0000	 4500 0045 09cd 0000 4006 eccc c0a8 0165	E..E....@......e
0x0010	 c0a8 0164 8a4a 046f 571e d992 fe17 b9d0	...d.J.oW.......
0x0020	 5018 ffff a833 0000 5241 574d 4f55 5345	P....3..RAWMOUSE

15:33:07.007871 IP X1000.35402 > SAM.1135: P 163800:163829(29) ack 28756 win 65535
0x0000	 4500 0045 09ce 0000 4006 eccb c0a8 0165	E..E....@......e
0x0010	 c0a8 0164 8a4a 046f 571e d992 fe17 b9d0	...d.J.oW.......
0x0020	 5018 ffff a833 0000 5241 574d 4f55 5345	P....3..RAWMOUSE
Suspected packet drop:
15:33:41.799568 IP X1000.35402 > SAM.1135: P 187571:187599(28) ack 32976 win 65535
0x0000	 4500 0044 0d3f 0000 4006 e95b c0a8 0165	E..D.?..@..[...e
0x0010	 c0a8 0164 8a4a 046f 571f 366d fe17 ca4c	...d.J.oW.6m...L
0x0020	 5018 ffff 49f5 0000 5241 574d 4f55 5345	P...I...RAWMOUSE

15:33:43.012563 IP X1000.35402 > SAM.1135: P 187571:187599(28) ack 32976 win 65535
0x0000	 4500 0044 0d40 0000 4006 e95a c0a8 0165	E..D.@..@..Z...e
0x0010	 c0a8 0164 8a4a 046f 571f 366d fe17 ca4c	...d.J.oW.6m...L
0x0020	 5018 ffff 49f5 0000 5241 574d 4f55 5345	P...I...RAWMOUSE
Suspected packet drop:
15:35:02.964386 IP X1000.35402 > SAM.1135: P 272961:272989(28) ack 47896 win 65535
0x0000	 4500 0044 1940 0000 4006 dd5a c0a8 0165	E..D.@..@..Z...e
0x0010	 c0a8 0164 8a4a 046f 5720 83fb fe18 0494	...d.J.oW.......
0x0020	 5018 ffff cd15 0000 5241 574d 4f55 5345	P.......RAWMOUSE

15:35:04.025814 IP X1000.35402 > SAM.1135: P 272961:272989(28) ack 47896 win 65535
0x0000	 4500 0044 1942 0000 4006 dd58 c0a8 0165	E..D.B..@..X...e
0x0010	 c0a8 0164 8a4a 046f 5720 83fb fe18 0494	...d.J.oW.......
0x0020	 5018 ffff cd15 0000 5241 574d 4f55 5345	P.......RAWMOUSE

8.AmigaOS4:Documentation/Roadshow/TCP-Dump> tcpdump >PIPE:logit -X port 35402
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on PPC440EP_ETH, link-type EN10MB (Ethernet), capture size 68 bytes
62858 packets captured
63666 packets received by filter
0 packets dropped by kernel

10.AmigaOS4:Documentation/Roadshow/TCP-Dump> rx drop.rx 
Suspected packet drop:
15:30:48.039912 IP amiga.1135 > X1000.35402: P 7756:7761(5) ack 43297 win 65535
0x0000	 4500 002d e2cb 0000 4006 13e6 c0a8 0164	E..-....@......d
0x0010	 c0a8 0165 046f 8a4a fe17 6700 571d 0011	...e.o.J..g.W...
0x0020	 5018 ffff 3826 0000 5243 5644 00       	P...8&..RCVD.

15:30:49.264923 IP amiga.1135 > X1000.35402: P 7756:7761(5) ack 43297 win 65535
0x0000	 4500 002d e2cc 0000 4006 13e5 c0a8 0164	E..-....@......d
0x0010	 c0a8 0165 046f 8a4a fe17 6700 571d 0011	...e.o.J..g.W...
0x0020	 5018 ffff 3826 0000 5243 5644 00       	P...8&..RCVD.
Suspected packet drop:
15:33:01.906872 IP amiga.1135 > X1000.35402: P 28956:28961(5) ack 164543 win 65535
0x0000	 4500 002d f535 0000 4006 017c c0a8 0164	E..-.5..@..|...d
0x0010	 c0a8 0165 046f 8a4a fe17 b9d0 571e d9af	...e.o.J....W...
0x0020	 5018 ffff 0bb6 0000 5243 5644 00       	P.......RCVD.

15:33:03.343969 IP amiga.1135 > X1000.35402: P 28956:28961(5) ack 164543 win 65535
0x0000	 4500 002d f537 0000 4006 017a c0a8 0164	E..-.7..@..z...d
0x0010	 c0a8 0165 046f 8a4a fe17 b9d0 571e d9af	...e.o.J....W...
0x0020	 5018 ffff 0bb6 0000 5243 5644 00       	P.......RCVD.
Suspected packet drop:
15:33:38.044981 IP amiga.1135 > X1000.35402: P 33176:33181(5) ack 188313 win 65535
0x0000	 4500 002d f8dd 0000 4006 fdd3 c0a8 0164	E..-....@......d
0x0010	 c0a8 0165 046f 8a4a fe17 ca4c 571f 3689	...e.o.J...LW.6.
0x0020	 5018 ffff 9e5f 0000 5243 5644 00       	P...._..RCVD.

15:33:39.399521 IP amiga.1135 > X1000.35402: P 33176:33181(5) ack 188313 win 65535
0x0000	 4500 002d f8df 0000 4006 fdd1 c0a8 0164	E..-....@......d
0x0010	 c0a8 0165 046f 8a4a fe17 ca4c 571f 3689	...e.o.J...LW.6.
0x0020	 5018 ffff 9e5f 0000 5243 5644 00       	P...._..RCVD.
Suspected packet drop:
15:34:59.136572 IP amiga.1135 > X1000.35402: P 48096:48101(5) ack 273703 win 65535
0x0000	 4500 002d 05cc 0000 4006 f0e5 c0a8 0164	E..-....@......d
0x0010	 c0a8 0165 046f 8a4a fe18 0494 5720 8417	...e.o.J....W...
0x0020	 5018 ffff 1688 0000 5243 5644 00       	P.......RCVD.

15:35:00.478567 IP amiga.1135 > X1000.35402: P 48096:48101(5) ack 273703 win 65535
0x0000	 4500 002d 05cf 0000 4006 f0e2 c0a8 0164	E..-....@......d
0x0010	 c0a8 0165 046f 8a4a fe18 0494 5720 8417	...e.o.J....W...
0x0020	 5018 ffff 1688 0000 5243 5644 00       	P.......RCVD.
Suspected packet drop:
15:35:58.704566 IP amiga.1135 > X1000.35402: P 56496:56501(5) ack 322395 win 65535
0x0000	 4500 002d 0d25 0000 4006 e98c c0a8 0164	E..-.%..@......d
0x0010	 c0a8 0165 046f 8a4a fe18 2564 5721 424b	...e.o.J..%dW!BK
0x0020	 5018 ffff 3783 0000 5243 5644 00       	P...7...RCVD.

15:36:00.049631 IP amiga.1135 > X1000.35402: P 56496:56501(5) ack 322395 win 65535
0x0000	 4500 002d 0d26 0000 4006 e98b c0a8 0164	E..-.&..@......d
0x0010	 c0a8 0165 046f 8a4a fe18 2564 5721 424b	...e.o.J..%dW!BK
0x0020	 5018 ffff 3783 0000 5243 5644 00       	P...7...RCVD.
Suspected packet drop:
15:38:45.661055 IP amiga.1135 > X1000.35402: P 79771:79776(5) ack 459322 win 65535
0x0000	 4500 002d 2155 0000 4006 d55c c0a8 0164	E..-!U..@..\...d
0x0010	 c0a8 0165 046f 8a4a fe18 804f 5723 592a	...e.o.J...OW#Y*
0x0020	 5018 ffff c5b6 0000 5243 5644 00       	P.......RCVD.

15:38:46.738284 IP amiga.1135 > X1000.35402: P 79771:79776(5) ack 459322 win 65535
0x0000	 4500 002d 2156 0000 4006 d55b c0a8 0164	E..-!V..@..[...d
0x0010	 c0a8 0165 046f 8a4a fe18 804f 5723 592a	...e.o.J...OW#Y*
0x0020	 5018 ffff c5b6 0000 5243 5644 00       	P.......RCVD.

_________________
BroadBlues On Blues BroadBlues On Amiga Walker Broad

@broadblues

I'm sorry, I did not look at this closely enough yet. Is this exactly how you invoke recv() in your code?

Quote:

TEXT buffer[512]; if( ISocket->recv(sock,buffer,sizeof(buffer),0) == -1) { DropClient(server,sock); return FALSE; }

I am asking because the send() counterpart in the code does not indicate how much data the receiver will need to read in order to obtain a complete protocol unit.

TCP BSD sockets act like FIFOs (pipes). If you call send(socket,buffer,10,0) then exactly ten bytes will be sent, and no "end of record" marker will be included. This means that when the receiver calls n=recv(socket,buffer,256,0) it may not just pick up the ten bytes sent by the sender, it might read more than that, if there is more unread data waiting in the network reception buffer which the socket connects to.

And it gets weirder still. That n=recv(socket,buffer,256,0) call above, it might not return ten bytes, it might return 1. When you tell recv() to read 256 bytes, you are actually telling it "please store up to 256 bytes in the buffer provided and hang around until you can store at least 1 byte in that buffer". recv() will return 0 if the sender closed the connection; from that point on every recv() call you make on that socket will keep returning 0.

In your code you seem to be catching only the error case (recv() returning -1), which is not sufficient if you want to read each single protocol unit without losing anything.

A more robust recv() wrapper function could look as follows:

ssize_t
recv_all(int sock,void * buffer,ssize_t buffer_size,int flags,int *error_ptr)
{
char * data = buffer;
ssize_t num_bytes_read;
ssize_t len;

num_bytes_read = 0;

(*error_ptr) = 0;

while(num_bytes_read < buffer_size)
{
len = recv(sock,data,(size_t)(buffer_size - num_bytes_read),flags);
if(len == -1)
{
(*error_ptr) = errno;

if(num_bytes_read == 0)
num_bytes_read = -1;

break;
}

if(len == 0)
break;

data += len;
num_bytes_read += len;
}

return(num_bytes_read);
}

This function will return 0 on EOF, -1 if an error occured before even one byte could be read, and otherwise the number of bytes read, which can be smaller than the number requested. If that number is smaller than what was requested, check what's in the variable you passed by reference as the last parameter: there might have been an error (non-zero value in the error variable), or the connection was closed before all the requested data could be read (error variable is zero).

Your current protocol does not indicate how long each protocol unit is that is committed to the wire. Each unit is terminated by a NUL byte, which suggests that you may have to call recv() for a single byte until you find the NUL byte which separates the individual records. This would yield poor performance, though.

A simple change would be to add a length field in front of each record you transmit. The receiver would then first read the length field, verify that the length is sound, proceed to the read exactly as many bytes as indicated, and check if the number of bytes read exactly matches the length given (error/EOF handling to follow).

Last edited by olsen on 07-Sep-2014 at 08:05 AM.
Last edited by olsen on 07-Sep-2014 at 08:05 AM.

@broadblues

Hang on, tcpdump output is notoriously terse and hard to read. You might want to tell tcpdump to capture and decode more than just the header and the first few bytes of each packet using the "-s" option, as in "tcpdump -s 65535". Next, you might want tell tcpdump to decode more of the header, using the "-vv" or "-vvv" options, as in "tcpdump -vvv".

I would recommend that you change your protocol to include sequence numbers, so that you don't have to rely upon TCP header information to tell you whether something is missing or not. This when the "-s" option will come in handy, allowing you to see on the spot whether the protocol sequence numbers increase in subsequent packets, or if there is something being repeated.

Incidentally, you don't have to call tcpdump over and over again. It may be sufficient to capture a single session and store that to disk using the "-w" option (lower case 'w'), as in "tcpdump -s 65535 -w capture-file.pcap". The resulting file can then be reread by tcpdump (or fancier tools such as Wireshark) and printed with more decoding options enabled using the "-r" option as in "tcpdump -vvv -r capture-file.pcap"

Last edited by olsen on 07-Sep-2014 at 08:02 AM.

@olsen

Quote:

TCP BSD sockets act like FIFOs (pipes). If you call send(socket,buffer,10,0) then exactly ten bytes will be sent, and no "end of record" marker will be included. This means that when the receiver calls n=recv(socket,buffer,256,0) it may not just pick up the ten bytes sent by the sender, it might read more than that, if there is more unread data waiting in the network reception buffer which the socket connects to.

I dealt with the issue of packets building up at the client or server incoming buffer by making my protocol synchronous, every packet gets a responce, be it a simple RCVD or a more detailed packet with info. No futher packets are sent until the response is received, if it never arrives at the client, it will disconnet from the sever and attempt a reconnect up to a retry limit (thus dealing with a reboot at the sever) . Like wise if a response never arrives at the server (or there is an error) it drops the client, and waits for new client connections (thus dealing with a reboot at the client).

Quote:

And it gets weirder still. That n=recv(socket,buffer,256,0) call above, it might not return ten bytes, it might return 1. When you tell recv() to read 256 bytes, you are actually telling it "please store up to 256 bytes in the buffer provided and hang around until you can store at least 1 byte in that buffer". recv() will return 0 if the sender closed the connection; from that point on every recv() call you make on that socket will keep returning 0.

I hadn't taken this into account in the main loop. In practice it's not happening, perhaps due to fact I'm working over the LAN and also my protocol units are short (30 bytes max). I can see though that perhaps I should take care of the possibilty for maximum robustness.

I have taken it into account with my ClipBoard sharing code where I send a "CLIPBOARD %lu" to say the next packet will be a special "raw clipboard" data packet of given size. Then read that into the clipboard.device in a loop.


I tried your recv_all code example (just modified to use ISocket->recv() rather than recv() but it blocks after the first read.

_________________
BroadBlues On Blues BroadBlues On Amiga Walker Broad

@olsen

Quote:

A simple change would be to add a length field in front of each record you transmit. The receiver would then first read the length field, verify that the length is sound, proceed to the read exactly as many bytes as indicated, and check if the number of bytes read exactly matches the length given (error/EOF handling to follow).

I rejigged my protocol to add a 1 byte sized size field at the begining (current max message size is only 32 bytes so no need for more). So now as suggested I read the first byte to check message length, and then use your recv_all() style function to ensure the complete message is read. It (recv_all() no longer blocks as it get exactly the right size now.

I added a bit of code to try an ease the effects of a tcp retry by discarding the agreagate mouse move data if the queue exceeds twice the normal maximum queue (as this is pretty indicator of a problem) and send x = 0 y = 0 instead, so the mouse sticks but doesn't jump after the stick.

_________________
BroadBlues On Blues BroadBlues On Amiga Walker Broad

@broadblues

Quote:

But note whilst mouse movements might be discardable, the other events mouse button up / down , keyboard up down

You can transferee the state of buttons (1 = Up, 0 = Down) instead of the event of buttons. That way if one package is lost its not important. (In the case of TCP it will retransmit the packages, and thats maybe not needed.)

And also you can encode it simple for example 10 x 0, 1 x 1, 20 x 0 , and so on. That way don't need to transferee a lot more information then a normal event.

Quote:

mouse wheel etc et are more critical.

Well I don't know about mouse wheel, but with some imagination, it should be possible to do some how.

Quote:

AS far as I know synergy uses TCP

TCP is normally used when data has to be synchronous and when there can't be data lose.

There for UDP is often used for Audio streams, like VoIP where keeping up is more important then some data lose, any how if audio is package is missing, some audio information.

UDP packages have less storage overhead as they do not have Package Sequence numbers and handshake stuff.

Quote:

AS far as I know synergy uses TCP

Yes easy to fall into habits

Last edited by NutsAboutAmiga on 08-Sep-2014 at 06:43 AM.
Last edited by NutsAboutAmiga on 08-Sep-2014 at 06:41 AM.
Last edited by NutsAboutAmiga on 08-Sep-2014 at 06:34 AM.

_________________
http://lifeofliveforit.blogspot.no/
Facebook::LiveForIt Software for AmigaOS

@broadblues

Quote:

broadblues wrote: @olsen Quote: TCP BSD sockets act like FIFOs (pipes). If you call send(socket,buffer,10,0) then exactly ten bytes will be sent, and no "end of record" marker will be included. This means that when the receiver calls n=recv(socket,buffer,256,0) it may not just pick up the ten bytes sent by the sender, it might read more than that, if there is more unread data waiting in the network reception buffer which the socket connects to. I dealt with the issue of packets building up at the client or server incoming buffer by making my protocol synchronous, every packet gets a responce, be it a simple RCVD or a more detailed packet with info. No futher packets are sent until the response is received, if it never arrives at the client, it will disconnet from the sever and attempt a reconnect up to a retry limit (thus dealing with a reboot at the sever) . Like wise if a response never arrives at the server (or there is an error) it drops the client, and waits for new client connections (thus dealing with a reboot at the client).

This still won't save you if the network traffic is broken up, for whatever reason. Packets do get fragmented and reassembled, even in a LAN. Consider fixing up the recv() behaviour just to avoid mixing one more source of random irritation into the design, which will make debugging harder.

Quote:

Quote: And it gets weirder still. That n=recv(socket,buffer,256,0) call above, it might not return ten bytes, it might return 1. When you tell recv() to read 256 bytes, you are actually telling it "please store up to 256 bytes in the buffer provided and hang around until you can store at least 1 byte in that buffer". recv() will return 0 if the sender closed the connection; from that point on every recv() call you make on that socket will keep returning 0. I hadn't taken this into account in the main loop. In practice it's not happening, perhaps due to fact I'm working over the LAN and also my protocol units are short (30 bytes max). I can see though that perhaps I should take care of the possibilty for maximum robustness.

You should. The API design of recv() does not let you off the hook, just because you got lucky so far.

Quote:

I have taken it into account with my ClipBoard sharing code where I send a "CLIPBOARD %lu" to say the next packet will be a special "raw clipboard" data packet of given size. Then read that into the clipboard.device in a loop.

The length field should lead each and every packet you send, and the receiver should not depend upon state information to be carried around from one or more packets received before.

For maximum simplicity, copy the length into a 16 bit word and prepend it to the message as is, and not encoded as ASCII text.
Quote:

I tried your recv_all code example (just modified to use ISocket->recv() rather than recv() but it blocks after the first read.

I'm relatively confident that my code is doing the right thing, and it's simple enough to verify (well, I didn't have a test harness for it). If it's blocking where previously calling recv() didn't block, it means that this needs looking into.

The recv_all() function is not a drop-in replacement for recv(). It needs to know exactly how much data must be read because it will block until everything you asked it to read has been received. Your code needs to be rewritten in order to allow for recv_all() to work.

I would suggest you put a length field in front of each message you send, use recv_all() to read that length field, then read the following message, and see where that leads you.

The thing is, right now you can't observe that much about the message exchange, and the contents of the message, even with the help of tcpdump, and you report dropped packets in a LAN, which is very rare thing.

It's possible that your network isn't dropping packets, but your processing of the messages is not as robust as you may expect. My advice would be to make the code compliant with the best practices for dealing with TCP BSD sockets and the somewhat quirky behaviour of recv(). These peculiarities need to be covered. You can't expect this to somehow work out, the API is not your friend.

Last edited by olsen on 08-Sep-2014 at 07:14 AM.

@broadblues

Quote:

broadblues wrote: @olsen Quote: A simple change would be to add a length field in front of each record you transmit. The receiver would then first read the length field, verify that the length is sound, proceed to the read exactly as many bytes as indicated, and check if the number of bytes read exactly matches the length given (error/EOF handling to follow). I rejigged my protocol to add a 1 byte sized size field at the begining (current max message size is only 32 bytes so no need for more). So now as suggested I read the first byte to check message length, and then use your recv_all() style function to ensure the complete message is read. It (recv_all() no longer blocks as it get exactly the right size now.

That sounds much better For the sake of paranoia, do you check if a message exceeds 255 bytes before you commit it to the wire?
Quote:

I added a bit of code to try an ease the effects of a tcp retry by discarding the agreagate mouse move data if the queue exceeds twice the normal maximum queue (as this is pretty indicator of a problem) and send x = 0 y = 0 instead, so the mouse sticks but doesn't jump after the stick.

OK, the protocol should be more robust now.

Do you still see packets dropping?

@broadblues

Technically I can't say anything, just wanted to let you know about Aremote. It is a classic tool which I use between an uA1 and a A1200 to share mouse and keyboard.
It is working a bit jaggy but have not experienced any extra delay while moving the pointer.

Hope you will solve the problem, that tool would be handy!

@Lazi

Quote:

Technically I can't say anything, just wanted to let you know about Aremote. It is a classic tool which I use between an uA1 and a A1200 to share mouse and keyboard. It is working a bit jaggy but have not experienced any extra delay while moving the pointer.

Interesting. I had a look didn't actually try it out, as it had source included I browsed through to compare. Bigest difference all it's networking code is encapsulated by amarquee.library, so no real way to compare that.

IN terms of envent capture it works similarly except that it sends the events from the commodity to a msgport and I add to amutext protected list.

In terms of adding events on the client side it adds them AddIEvent() from commodities .library this will be why it's 'a bit jaggy' as the update rate seems to be about 10 frames a second here with mousemoves agregated.

Quote:

Hope you will solve the problem, that tool would be handy!

Whether it's a problem or not depends on what you want to do. I'm using it to reply to this message, cutting and pasting from the quoted section with the mouse, typing etc etc with no issues at all. (if it would only fix my typing skills! )

It's more of an issue when dealing with a gfx app or other situation where you need more precise mouse movements.

_________________
BroadBlues On Blues BroadBlues On Amiga Walker Broad

@olsen

Quote:

Do you still see packets dropping?

Unfotunatly yes.

_________________
BroadBlues On Blues BroadBlues On Amiga Walker Broad