@olsen

Quote:

The tool I had in mind did not log packets, it allowed you to create packets, with a GUI, and send them to file systems and handlers. While you could always roll your own code to do that, such a tool can be really neat

Yes it sounds neat but what about at the other end with the actual code? That's where I wanted to know what was going on.

My tool was more simpler, it didn't bother to generate any packets, it left that up to the system by putting it in the real world. The real world accorrding to what programs you tested it with.

@olsen

Quote:

I have just updated the client code again.

BTW while I think of it a friend has been having trouble using Roadshow with his router than I flashed dd-wrt onto. IIRC I could still use DHCP settings but had to set the gateway to static.

Which reminds me the IP address has changed since.

@Hypex

Quote:

Hypex wrote: @olsen Quote: The tool I had in mind did not log packets, it allowed you to create packets, with a GUI, and send them to file systems and handlers. While you could always roll your own code to do that, such a tool can be really neat Yes it sounds neat but what about at the other end with the actual code? That's where I wanted to know what was going on.

I finally figured out the name of the tool: PickPacket. You could find it on Fish Disk #227.

As for file systems and handlers, there are few examples. Some you could find on the old Fish Disks.

The last bigger file system I released the source code for was "smbfs". It's a peculiar design because it runs as a shell command or Workbench tool rather requiring you to use the Mount command. But there's an interesting bonus to using that design: you can run smbfs in a debugger and watch what it does.

@olsen

Quote:

So, long story short, don't try this at home. Wait for your modem and gateway router to become ready before you fire up your web browser.

I have made some tests and I have some difficulties to admit that the problem came from the brower (the SYNC and DATA led of the modem must be fixed before using the browser).

I have really waited after the modem completely SYNCed and DATAed to use the browser and I have seen the same wait state on some internet page - blue square forever on Ibrowse channel (I often use browser link one by one and allways the same).

I have rebooted the SAM after and re-use the browser and this time no DNS wait.

For me, apparently, have the modem SYNCed and DATAed before the startup encountering addnetinterface help a lot for browsing fine.

I haven't tested but from me memory, if I no wait to have modem completely ready with OWB and try to see a page, OWB keep without datas from internet forever....


Do you think, if the modem isn't completely ready and addnetinterface arrive on the startup, something couldn't run well with internet ????

_________________
BTW, what you have done for the amiga today ????

-A1200+Mediator+VooDoo3+060/50+96mo+SCSI-KIT
-SAM440EP-667mhz-on MapowerKC3000+AOS4.1

Amiga Docs Disks Preservation Project

@Mrodfr

Do not turn on your computer until after your modem is synchronized and receiving data.

_________________
Sam440ep-flex 733 MHz/1 GB RAM/Radeon 9250/AmigaOS4.1 Update 2
borked A1200/Blizzard1260+SCSI-IV/Z4+MediatorZIV/Deneb/Voodoo3/CatweaselMk3
more borked A1200/MBX1200z/Indivision
A500/clockport/RRNet
A600/A603

@Trev

until after ???

I must turn on my computer after my modem is synchronized and receiving data led activated.

Like that, I have no DNS lookup forever wait problem with Ibrowse or OWB.

As my modem is an old one (with the first ADSL technology), the amiga is booted before modem completely synchronized and/or data receiving activated.

I have just removed my static setting (on prefs/internet) to dynamic and check if that will change something -just to have a more standard setting)..

Last edited by Mrodfr on 12-Sep-2010 at 07:01 PM.

_________________
BTW, what you have done for the amiga today ????

-A1200+Mediator+VooDoo3+060/50+96mo+SCSI-KIT
-SAM440EP-667mhz-on MapowerKC3000+AOS4.1

Amiga Docs Disks Preservation Project

@Mrodfr

Yes, after:

1. Turn on modem.
2. Wait for modem to synchronize and receive data.*
3. Turn on Amiga.

Ce devrait être aussi facile qu'un, deux, trois.

Your modem behaves like a bridge (something that converts one medium to another) and is similar to a traditional POTS modem. Step 2 is analogous to a POTS modem's connection and training sequences. Unlike with a POTS modem, however, Roadshow has no control over the state of your DSL modem, and you must manage it separately.

* If your modem's DATA LED is connected to the LAN jack, i.e. the one your Amiga is connected to, then you only need to wait until the modem is synchronized.

Last edited by Trev on 12-Sep-2010 at 08:33 PM.
Last edited by Trev on 12-Sep-2010 at 08:22 PM.
Last edited by Trev on 12-Sep-2010 at 08:20 PM.
Last edited by Trev on 12-Sep-2010 at 08:20 PM.

_________________
Sam440ep-flex 733 MHz/1 GB RAM/Radeon 9250/AmigaOS4.1 Update 2
borked A1200/Blizzard1260+SCSI-IV/Z4+MediatorZIV/Deneb/Voodoo3/CatweaselMk3
more borked A1200/MBX1200z/Indivision
A500/clockport/RRNet
A600/A603

@Mrodfr

Quote:

Do you think, if the modem isn't completely ready and addnetinterface arrive on the startup, something couldn't run well with internet ????

Yes.

That's what I tried to tell you. When you access a web page, an internet access is being made before the page is even loaded. This access must complete successfully within 5-15 seconds (preferably faster), or the web page will not load.

You will not succeed if you try to use your Amiga to access the internet before your connection to the internet is completely operational.

@olsen

Quote:

That's what I tried to tell you. When you access a web page, an internet access is being made before the page is even loaded. This access must complete successfully within 5-15 seconds (preferably faster), or the web page will not load.

It is clear now

All people do the same as me ??? (hum, surely, modern ADSL2+ synced and dataed more rapidly than my ADSL1).

_________________
BTW, what you have done for the amiga today ????

-A1200+Mediator+VooDoo3+060/50+96mo+SCSI-KIT
-SAM440EP-667mhz-on MapowerKC3000+AOS4.1

Amiga Docs Disks Preservation Project

@Mrodfr

Quote:

Mrodfr wrote: @olsen Quote: That's what I tried to tell you. When you access a web page, an internet access is being made before the page is even loaded. This access must complete successfully within 5-15 seconds (preferably faster), or the web page will not load. It is clear now All people do the same as me ??? (hum, surely, modern ADSL2+ synced and dataed more rapidly than my ADSL1).

Not everybody turns his ADSL modem/router off. Some people, including myself, keep their networking gear permanently turned on. If the devices are reasonably modern, the total power consumption should be negligible, and if your ISP contract has a flat rate for traffic, there should be no extra cost at keeping the internet connection permanently active.

@olsen

Thanks a lot for all your informations

I know now: let the modem completely found his sync and datas and launch the Amiga after.

I have now best result with static adress and DHCP (dynamic not work well).

Do you have some advices about overwriting some default settings for roadshow, setings located on env/roadshow, like: MTU, sendspace or recvspace,....

I have recently played with a PC tool who give best result about send and recvspace and the values change depending of the internet speed.

_________________
BTW, what you have done for the amiga today ????

-A1200+Mediator+VooDoo3+060/50+96mo+SCSI-KIT
-SAM440EP-667mhz-on MapowerKC3000+AOS4.1

Amiga Docs Disks Preservation Project

@Mrodfr

Research the following topics:

latency
path MTU discovery
TCP receive window
TCP window scaling
bandwidth delay product

Each application is also responsible for maintaining its own buffers, and bad buffer management can cause unexpected delays:

Nagle's algorithm
tcp no delay

olsen can speak to what Roadshow supports and what you can configure, but the topics above do apply to most IP stacks and TCP application protocols. You have no control over other people's configurations, so changing your own might not make a difference in all cases.

EDIT: This is the TCP/IP "Bible": TCP/IP Illustrated.

Last edited by Trev on 13-Sep-2010 at 07:41 PM.

_________________
Sam440ep-flex 733 MHz/1 GB RAM/Radeon 9250/AmigaOS4.1 Update 2
borked A1200/Blizzard1260+SCSI-IV/Z4+MediatorZIV/Deneb/Voodoo3/CatweaselMk3
more borked A1200/MBX1200z/Indivision
A500/clockport/RRNet
A600/A603

@Trev

Quote:

olsen can speak to what Roadshow supports and what you can configure, but the topics above do apply to most IP stacks and TCP application protocols. You have no control over other people's configurations, so changing your own might not make a difference in all cases.

If you are connecting to the internet through an ADSL modem, then the following may improve network performance:

roadshowcontrol save set tcp.do_timestamps 0
roadshowcontrol save set tcp.do_win_scale 0
roadshowcontrol save set tcp.mssdflt 1460
roadshowcontrol save set tcp.recvspace 65536
roadshowcontrol save set tcp.use_mssdflt_for_remote 0

Enter these commands in the shell, exactly as given above, and reboot your machine. Since the settings are saved in files stored in the "ENVARC:" directory you need to enter these commands only once. To undo the effect these settings have, just delete the "ENVARC:Roadshow" directory and reboot.

Quote:

EDIT: This is the TCP/IP "Bible": TCP/IP Illustrated.

Hum, big scary book, that is Not for the easily bored, and definitely not light reading...

@olsen

Thanks for the informations

For our informations, on the site speedguide.net, there are a tool called: SG TCP Optimizer (windowXP). http://www.speedguide.net/downloads.php

With an internet connection of 3,5 Mb (my house), this tool give:

MTU=1500 TCP receive window=256960
MTU=1492 TCP receive window=261360

for example with an internet connection of 10 mb:

MTU=1500 TCP receive window=513920
MTU=1492 TCP receive window=522720

What do you think about this " verry " high values for TCP receive window (recvspace) linked witht he speed of the internet connection ???

_________________
BTW, what you have done for the amiga today ????

-A1200+Mediator+VooDoo3+060/50+96mo+SCSI-KIT
-SAM440EP-667mhz-on MapowerKC3000+AOS4.1

Amiga Docs Disks Preservation Project

@Mrodfr

Quote:

Mrodfr wrote: @olsen Thanks for the informations For our informations, on the site speedguide.net, there are a tool called: SG TCP Optimizer (windowXP). http://www.speedguide.net/downloads.php With an internet connection of 3,5 Mb (my house), this tool give: MTU=1500 TCP receive window=256960 MTU=1492 TCP receive window=261360 for example with an internet connection of 10 mb: MTU=1500 TCP receive window=513920 MTU=1492 TCP receive window=522720 What do you think about this " verry " high values for TCP receive window (recvspace) linked witht he speed of the internet connection ???

I think this isn't very informative data.

The TCP receive window size is determined automatically as more and more data is being received without a transmission error being detected. The more data is being sent successfully, the less often the sender needs to be notified how much data has been sent so far.

This is what the TCP receive window size does: it is the number of bytes that may be received before the sender must be notified that all the data transmitted so far was received correctly. The larger the window size, the fewer notifications have to be sent, in relation to the total amount of data being sent. A larger window size causes the available bandwidth to be used more efficiently.

The MTU size is a different matter. Normally, Ethernet allows packets of up to 1500 bytes to be transmitted. But some devices do not allow for packets that large to be transmitted, which is why these packets have to be broken into smaller fragments before they can be sent. Breaking down the packets and reassembling them later takes its time. Therefore you would not want this to happen.

In Europe it is quite common to have ADSL modems talk to home networks through a protocol known as PPPoE. This protocol has a packet size limitation to the effect that packets larger than 1492 bytes will have to be broken up into fragments in order to be transmitted.

This is why the MTU settings have an effect here. The 1500 byte packets won't fit into what PPPoE uses for transport, causing additional overhead. But if you limit your packet sizes to 1492 bytes, then the packets will fit perfectly, incurring no additional overhead.

@olsen

Can the options be changed per interface or are they global?

Quote:

The TCP receive window size is determined automatically as more and more data is being received without a transmission error being detected. The more data is being sent successfully, the less often the sender needs to be notified how much data has been sent so far.

But haven't you told him to disable scaling with this:

roadshowcontrol save set tcp.do_win_scale 0

Quote:

The MTU size is a different matter. Normally, Ethernet allows packets of up to 1500 bytes to be transmitted. But some devices do not allow for packets that large to be transmitted, which is why these packets have to be broken into smaller fragments before they can be sent. Breaking down the packets and reassembling them later takes its time. Therefore you would not want this to happen

If Roadshow doesn't do path MTU discovery, then any packet destined for a remote network should have an MTU of 576. The name and documentation for tcp.use_mssdflt_for_remote are a little confusing:

Quote:

tcp.use_mssdflt_for_remote Controls if the TCP protocol should use a smaller maximum segment size value for packets sent to hosts which are not in the local network. This can be 1 (yes) or 0 (no).

The name implies the setting is tied to tcp.mssdflt. If tcp.use_mssdflt_for_remote is 1, does Roadshow use tcp.mssdflt or 536 (or less with RFC 1323 options) as the MSS?

Quote:

This is why the MTU settings have an effect here. The 1500 byte packets won't fit into what PPPoE uses for transport, causing additional overhead. But if you limit your packet sizes to 1492 bytes, then the packets will fit perfectly, incurring no additional overhead.

An MSS of 1460 implies that RFC 1323 options are disabled and the interface has an MTU of 1500. If the MTU can't be set at the interface level, wouldn't we want an MSS of 1452 for PPPoE? We would then want a receive window between 5808 and 65340 in multiples of 1452, depending on bandwidth requirements and latency to the furthest host.

Without path MTU discovery, I think the best general case would be to determine the maximum MTU between the host and the first hop router at the ISP using ping or another tool. If we assume PPPoE is in use, then these settings should be safe (but conservative):

roadshowcontrol save set tcp.do_timestamps 0
roadshowcontrol save set tcp.do_win_scale 0
roadshowcontrol save set tcp.mssdflt 1452
roadshowcontrol save set tcp.recvspace 65340
roadshowcontrol save set tcp.use_mssdflt_for_remote 0

(I'm assuming that tcp.use_mssdflt_for_remote = 0 doesn't restrict the MSS to minimum host requirements.)

EDIT: We might still run into trouble if the application's send or receive buffers are smaller than MSS*2 or if an application sends less than MSS*2 in data. MSS*4 is a safe minimum to offset delays in acknowledgements, assuming the host isn't extremely memory constrained. We can't control this in Roadshow unless there's a "tcp no delay" setting of some sort, assuming Roadshow uses Nagle's algorithm.

Trev

Last edited by Trev on 15-Sep-2010 at 05:17 PM.
Last edited by Trev on 15-Sep-2010 at 05:13 PM.

_________________
Sam440ep-flex 733 MHz/1 GB RAM/Radeon 9250/AmigaOS4.1 Update 2
borked A1200/Blizzard1260+SCSI-IV/Z4+MediatorZIV/Deneb/Voodoo3/CatweaselMk3
more borked A1200/MBX1200z/Indivision
A500/clockport/RRNet
A600/A603

@Trev

Quote:

Trev wrote: @olsen Can the options be changed per interface or are they global?

These are global variables affecting the behaviour of the TCP/IP stack itself.

Quote:

Quote: The TCP receive window size is determined automatically as more and more data is being received without a transmission error being detected. The more data is being sent successfully, the less often the sender needs to be notified how much data has been sent so far. But haven't you told him to disable scaling with this: roadshowcontrol save set tcp.do_win_scale 0

If I remember correctly, this option controls whether the window size can be changed to intervals greater than 65536 bytes. It does not disable window size scaling, it just keeps the maximum window size small.

I didn't come up with this setting or verify its effect. I believe it came out of one of those many TCP tuning guides. Maybe it's time to find out if it does anything useful in the first place

Quote:

Quote: The MTU size is a different matter. Normally, Ethernet allows packets of up to 1500 bytes to be transmitted. But some devices do not allow for packets that large to be transmitted, which is why these packets have to be broken into smaller fragments before they can be sent. Breaking down the packets and reassembling them later takes its time. Therefore you would not want this to happen If Roadshow doesn't do path MTU discovery, then any packet destined for a remote network should have an MTU of 576.

I'm not so sure about the requirement. If any relay along the path does not allow packets of 1500 bytes to pass, it's not the job of the ends of the TCP connection to adjust their packet sizes. The relay that won't let the 1500 bytes pass needs to tell the next relay to break the packet into fragments, or has to perform the fragmentation itself.

Path MTU discovery is, if I remember correctly, an optimization to avoid the overhead of packet fragmentation taking place on the path to the other end of the TCP connection. Its absence will not do great harm, it may just lead to worse overall performance. After all, if the fragmentation and reassembly has to be done for every packet coming through, it always means more work than sending the packets in the optimal size.

Or maybe it doesn't. This optimization was introduced some 15-20 years ago, and today's networks are very different animals. I reckon it will be difficult to even find one which will have to break 1500 byte packets into smaller fragments.

Quote:

The name and documentation for tcp.use_mssdflt_for_remote are a little confusing: Quote: tcp.use_mssdflt_for_remote Controls if the TCP protocol should use a smaller maximum segment size value for packets sent to hosts which are not in the local network. This can be 1 (yes) or 0 (no). The name implies the setting is tied to tcp.mssdflt. If tcp.use_mssdflt_for_remote is 1, does Roadshow use tcp.mssdflt or 536 (or less with RFC 1323 options) as the MSS?

If I remember correctly, the maximum segment size is set to a default value of 512 bytes, or something in that range, if the option is enabled (= 1) and the destination address of the TCP connection is not known to be in a local network.

If you disable this option, then the original interface MTU value will stick, leading to larger packets to be transmitted.

Quote:

Quote: This is why the MTU settings have an effect here. The 1500 byte packets won't fit into what PPPoE uses for transport, causing additional overhead. But if you limit your packet sizes to 1492 bytes, then the packets will fit perfectly, incurring no additional overhead. An MSS of 1460 implies that RFC 1323 options are disabled and the interface has an MTU of 1500. If the MTU can't be set at the interface level, wouldn't we want an MSS of 1452 for PPPoE? We would then want a receive window between 5808 and 65340 in multiples of 1452, depending on bandwidth requirements and latency to the furthest host.

Sounds good to me.

Quote:

Without path MTU discovery, I think the best general case would be to determine the maximum MTU between the host and the first hop router at the ISP using ping or another tool. If we assume PPPoE is in use, then these settings should be safe (but conservative): roadshowcontrol save set tcp.do_timestamps 0 roadshowcontrol save set tcp.do_win_scale 0 roadshowcontrol save set tcp.mssdflt 1452 roadshowcontrol save set tcp.recvspace 65340 roadshowcontrol save set tcp.use_mssdflt_for_remote 0 (I'm assuming that tcp.use_mssdflt_for_remote = 0 doesn't restrict the MSS to minimum host requirements.) EDIT: We might still run into trouble if the application's send or receive buffers are smaller than MSS*2 or if an application sends less than MSS*2 in data. MSS*4 is a safe minimum to offset delays in acknowledgements, assuming the host isn't extremely memory constrained. We can't control this in Roadshow unless there's a "tcp no delay" setting of some sort, assuming Roadshow uses Nagle's algorithm. Trev

Hm... clearly some testing would be required here. Many of the tricks of the trade in TCP tuning came about when the state of the art was the early to mid-1990'ies internet, and the typical TCP/IP stack was derived from something along the lines of BSD Net/2.

Given today's network technology, I reckon that the tuning measures recommended will have little impact. The most you can do today probably involves adjusting the send/receive buffer default sizes and keeping your default packet sizes as high as possible. Many TCP/IP stack default settings are still catering to the lower end of the scale where this is no longer required for proper operation.

Beyond some point you probably should stop caring about fiddling with the TCP/IP stack settings and just assume that your gateway router will do all the heavy lifting, involving packet fragmentation, reassembly and scrubbing.

Last edited by olsen on 16-Sep-2010 at 08:37 AM.

@olsen

Quote:

I finally figured out the name of the tool: PickPacket.

Thanks. Sounds good.

Quote:

As for file systems and handlers, there are few examples.

Should have uploaded mine.

Quote:

It's a peculiar design because it runs as a shell command or Workbench tool rather requiring you to use the Mount command.

I suppose when you run it as an internet application or like a daemon you can't exactly mount it.

Quote:

But there's an interesting bonus to using that design: you can run smbfs in a debugger and watch what it does.

Aha! About the only filesystem you could do that with. That's exactly what my program does, but only with handlers. It only does simple startup messages.

@olsen

Quote:

I didn't come up with this setting or verify its effect. I believe it came out of one of those many TCP tuning guides. Maybe it's time to find out if it does anything useful in the first place.

Well, it should be useful for very fat, very fast networks where receive windows can be in the megabytes. I've never observed a receive window at or near the maximum (1 GB?), but I'm sure there's an implementation out there somewhere. (EDIT: If it weren't an issue somewhere, we wouldn't have jumbo Ethernet frames with an MTU of 9000.) I think the most common default window size you'll find is 17520 bytes, with scaling enabled to augment performance.

Quote:

Or maybe it doesn't. This optimization was introduced some 15-20 years ago, and today's networks are very different animals. I reckon it will be difficult to even find one which will have to break 1500 byte packets into smaller fragments.

I don't think they're all that different, at least not in general purpose LANs.

We know the PPPoE MTU is less than 1500, and that's a pretty common example. The 802.11 MTU is 2304 bytes, but I think most driver implementations use 1500 by default.

Quote:

Beyond some point you probably should stop caring about fiddling with the TCP/IP stack settings and just assume that your gateway router will do all the heavy lifting, involving packet fragmentation, reassembly and scrubbing.

I can attest to that being a bad assumption for applications that process millions transactions over a single application-level request. Even a small delay, say 600 microseconds, can add hours to overall processing time. (That wasn't an arbitrary example. It's based on real world experience with geologically separate data centers and gigabit MAN links.) EDIT: It's also quite easy to bring about a situation that suffers from a host's delayed acknowledgment algorithm.

I doubt we'll ever see an Amiga (and Roadshow) in a large scale production environment again, so it probably doesn't matter. And as you've said before, we're likely to see bottlenecks in the design of SANA-II before the IP stack becomes an issue.

Last edited by Trev on 16-Sep-2010 at 07:59 PM.
Last edited by Trev on 16-Sep-2010 at 07:57 PM.
Last edited by Trev on 16-Sep-2010 at 07:55 PM.

_________________
Sam440ep-flex 733 MHz/1 GB RAM/Radeon 9250/AmigaOS4.1 Update 2
borked A1200/Blizzard1260+SCSI-IV/Z4+MediatorZIV/Deneb/Voodoo3/CatweaselMk3
more borked A1200/MBX1200z/Indivision
A500/clockport/RRNet
A600/A603

@Trev

Quote:

Trev wrote: @olsen Quote: I didn't come up with this setting or verify its effect. I believe it came out of one of those many TCP tuning guides. Maybe it's time to find out if it does anything useful in the first place. Well, it should be useful for very fat, very fast networks where receive windows can be in the megabytes. I've never observed a receive window at or near the maximum (1 GB?), but I'm sure there's an implementation out there somewhere. (EDIT: If it weren't an issue somewhere, we wouldn't have jumbo Ethernet frames with an MTU of 9000.) I think the most common default window size you'll find is 17520 bytes, with scaling enabled to augment performance.

It would be nice if the Amiga were to play in the league where such data throughput issues are taken seriously

For now I guess we're stuck with home networks, and maybe the occasional corporate network in which the Amiga at least should not cause any distress.

Quote:

Quote: Beyond some point you probably should stop caring about fiddling with the TCP/IP stack settings and just assume that your gateway router will do all the heavy lifting, involving packet fragmentation, reassembly and scrubbing. I can attest to that being a bad assumption for applications that process millions transactions over a single application-level request. Even a small delay, say 600 microseconds, can add hours to overall processing time. (That wasn't an arbitrary example. It's based on real world experience with geologically separate data centers and gigabit MAN links.) EDIT: It's also quite easy to bring about a situation that suffers from a host's delayed acknowledgment algorithm.

At this scale the most minute optimizations become important. But not necessarily in a home network, with an Amiga

The choice and configuration of a gateway router can make noticeable, even significant differences. A couple of years back (in 2005 or 2006) I was preparing to upgrade the OpenBSD firewall setup we use in my company, and I ran some tests at home using an old PC (built around 1999/2000). When I had this OpenBSD installation up and running, out of curiousity I added PPPoE support and temporarily replaced my gateway router with the OpenBSD system.

That temporary change made a difference, and a noticeable one, too. The OpenBSD system was much better at packet reassembly, for a start. Also, its use of send/receive buffers for network I/O was better than what my old gateway router could pull off. Unlike a simple gateway router, the OpenBSD packet filter did not just perform NAT and packet forwarding, it also actively reassembled fragments, cleaning up the TCP streams passing through it.