Click Here
home features news forums classifieds faqs links search
6071 members 
Amiga Q&A /  Free for All /  Emulation /  Gaming / (Latest Posts)
Login

Nickname

Password

Lost Password?

Don't have an account yet?
Register now!

Support Amigaworld.net
Your support is needed and is appreciated as Amigaworld.net is primarily dependent upon the support of its users.
Donate

Menu
Main sections
» Home
» Features
» News
» Forums
» Classifieds
» Links
» Downloads
Extras
» OS4 Zone
» IRC Network
» AmigaWorld Radio
» Newsfeed
» Top Members
» Amiga Dealers
Information
» About Us
» FAQs
» Advertise
» Polls
» Terms of Service
» Search

IRC Channel
Server: irc.amigaworld.net
Ports: 1024,5555, 6665-6669
SSL port: 6697
Channel: #Amigaworld
Channel Policy and Guidelines

Who's Online
29 crawler(s) on-line.
 130 guest(s) on-line.
 0 member(s) on-line.



You are an anonymous user.
Register Now!
 BigD:  14 mins ago
 OlafS25:  16 mins ago
 kolla:  35 mins ago
 amigakit:  44 mins ago
 NutsAboutAmiga:  47 mins ago
 Gunnar:  1 hr 25 mins ago
 Frank:  1 hr 32 mins ago
 MickJT:  2 hrs 11 mins ago
 A1200:  2 hrs 35 mins ago
 outlawal2:  3 hrs 9 mins ago

/  Forum Index
   /  Amiga OS4 Software
      /  What's with all these program needing WGet?
Register To Post

Goto page ( Previous Page 1 | 2 | 3 Next Page )
PosterThread
Hypex 
Re: What's with all these program needing WGet?
Posted on 18-Feb-2012 14:24:26
#21 ]
Elite Member
Joined: 6-May-2007
Posts: 11180
From: Greensborough, Australia

@Chris_Y

Oh no another dependence to install!

Looks like good solution. So how about a library version?

 Status: Offline
Profile     Report this post  
ChaosLord 
Re: What's with all these program needing WGet?
Posted on 18-Feb-2012 15:10:03
#22 ]
Cult Member
Joined: 4-Apr-2005
Posts: 782
From: Houston, Texas USA

@Hypex


Howzabout somebody compiles a pure residentable version of WGET?

That way programs do not have to call an external program any more. They can call an internal command.

_________________
Wanna try a wonderfull magical Amiga strategy game?
Total Chaos AGA

 Status: Offline
Profile     Report this post  
Xenic 
Re: What's with all these program needing WGet?
Posted on 18-Feb-2012 15:37:21
#23 ]
Super Member
Joined: 2-Feb-2004
Posts: 1246
From: Pennsylvania, USA

@Hypex
Quote:
I'm seeing a number of programs that use WGet. And I wonder why they are programmed to need an external CLI command just to load some data off the Internet?

Probably because it can be the quickest and easiest way to program in that kind of functionality. A lot of Amiga shared libraries are external. Why load a library just to call one simple function in that library instead of spending the day duplicating the function in your own code? Why would you use datatyes to play a sound when you can use ahi directly? Maybe to avoid duplicating existing programming and concentrate on your own program.
Quote:
Executable binaries shouldn't be requiring a 3rd party tool for this kind of thing. Libraries should be used for this. For instance OS4 has the Roadshow API and also a BSDSocket API like with other Amiga Internet stacks. THat would be the proper way to do it.

Does that include 3rd party SOBJs? They all get loaded external to the main program.
Quote:
Okay so that would require some learning. But if it is only a simple TCP transaction then the next best thing is a DOS handler. OS4 provides a TCP: device just like Miami did. Why isn't this being used more often?

Documentation?? Search the SDK documentation and user documentation for TCP: or tcp-handler. Do you think a programmer with limited experience could get the job done based on a short paragraph of vague explanation and a single command line example??
Quote:
The worse way about it is to use a command on disk and I amn seeing this a lot. Okay I have WGet installed but I don't want running commodities and applications loading in a command everytime they check the internet!

Loading WGet every time the Internet is checked would be unnecessary if OS4 included a reentrant (pure) version of WGet or Curl (or similar command) that could be made "RESIDENT" using the Resident command in the startup-sequence. A resident command is loaded from disk only once like a shared library but is simpler and easier to use.

If excess disk activity bothers you then you probably need to run Snoopy or SnoopDOS while some of the ported Linux programs are running. A lot of them I've checked will open and close the same files over and over again for no apparent reason. Some of them open and close libraries or devices numerous times instead of opening once at the beginning of the program and closing them on exit. Some native Amiga programs seem to do the same thing but not to the same extent.

_________________
X1000 with 2GB memory & OS4.1FE

 Status: Offline
Profile     Report this post  
NutsAboutAmiga 
Re: What's with all these program needing WGet?
Posted on 18-Feb-2012 15:45:47
#24 ]
Elite Member
Joined: 9-Jun-2004
Posts: 12795
From: Norway

@Xenic

Curl is the same as TCP: handler.

Type TCP:80/www.amigawold.net/index.htm

should download the index.page, and dump it in shell.

Curl “http://www.amigaworld.net/index.htm”

will download text and display it in terminal.

_________________
http://lifeofliveforit.blogspot.no/
Facebook::LiveForIt Software for AmigaOS

 Status: Offline
Profile     Report this post  
broadblues 
Re: What's with all these program needing WGet?
Posted on 18-Feb-2012 16:24:31
#25 ]
Amiga Developer Team
Joined: 20-Jul-2004
Posts: 4446
From: Portsmouth England

@NutsAboutAmiga

Quote:

Curl is the same as TCP: handler.

Type TCP:80/www.amigawold.net/index.htm

should download the index.page, and dump it in shell.

Curl ?http://www.amigaworld.net/index.htm?

will download text and display it in terminal.


That'scompletely wrong.

It's not nearly so simple as the following test shows....

in python


>>> fileh = open("TCP:www.amigaworld.net/80","r+")
>>> fileh.write("GET index.html HTTP/1.0\nhost: www.amigaworld.net\n\n")
>>> fileh.read()
'HTTP/1.1 400 Bad Request\r\nDate: Sat, 18 Feb 2012 16:17:55 GMT\r\nServer: \r\n
Last-Modified: Mon, 11 Apr 2011 18:38:46 GMT\r\n
ETag: "284015-3f8-4a0a8e2bf9980"\r\nAccept-Ranges: bytes\r\nContent-Length: 1016\r\n
Vary: Accept-Encoding\r\nConnection: close\r\nContent-Type: text/html; charset=iso-8559-1\r\n\r\n\n\n400 Bad Request\'
>>>



Note I cropped the response as it would have messed up the forum.

Now I used to maintain a webrowser I stil got it wrong!

I'm wandering if I need the \r as well....

[edit]
The error was that I was leavng the leading "/" of of the path so the result should be


>>> fileh = open("TCP:www.amigaworld.net/80","r+")
>>> fileh.write("GET /index.html HTTP/1.0\r\nHost: www.amigaworld.net\r\n\r\n")
>>> fileh.read()
'HTTP/1.1 200 OK\r\nDate: Sat, 18 Feb 2012 16:33:32 GMT\r\nServer: \r\nLast-Modified: Mon, 11 Apr 2011 18:54:30 GMT\r\n .....



So the server was sending the wrong error message....

I think this shows quite nice;y how it better to use wget etc from scripts if all you want is to fetch a single file from the net.

Using from binaries is another matter, then you should be using libcurl or simlar, if it's not appropriate to write your own http layer

Last edited by broadblues on 18-Feb-2012 at 04:38 PM.
Last edited by broadblues on 18-Feb-2012 at 04:35 PM.
Last edited by broadblues on 18-Feb-2012 at 04:25 PM.

_________________
BroadBlues On Blues BroadBlues On Amiga Walker Broad

 Status: Offline
Profile     Report this post  
NutsAboutAmiga 
Re: What's with all these program needing WGet?
Posted on 18-Feb-2012 17:45:32
#26 ]
Elite Member
Joined: 9-Jun-2004
Posts: 12795
From: Norway

@broadblues

He he I forget you need ask for index.htm page

Last edited by NutsAboutAmiga on 18-Feb-2012 at 05:45 PM.

_________________
http://lifeofliveforit.blogspot.no/
Facebook::LiveForIt Software for AmigaOS

 Status: Offline
Profile     Report this post  
Chris_Y 
Re: What's with all these program needing WGet?
Posted on 18-Feb-2012 18:10:07
#27 ]
Elite Member
Joined: 21-Jun-2003
Posts: 3203
From: Beds, UK

@Hypex

A library version would be something completely different, for a start you'd lose the ease of just opening and reading files. For example, in ARexx with http-handler you can do this to show a file:
Quote:
/* http arexx example */
open(fh, "http://www.amigaworld.net", "r")
do until eof(fh)
say readln(fh)
end
close(fh)


ie. it's identical to what you'd do to acces t:usb.log or any other local file you might want to parse.

Using TCP: is good too, but when you end up with chunked encoding, authentication , SSL, proxy servers or other complications, it isn't really that easy.

Last edited by Chris_Y on 18-Feb-2012 at 06:13 PM.

_________________
"Miracles we do at once, the impossible takes a little longer" - AJS on Hyperion
Avatar is Tabitha by Eric W Schwartz

 Status: Offline
Profile     Report this post  
Tomas 
Re: What's with all these program needing WGet?
Posted on 18-Feb-2012 18:21:21
#28 ]
Elite Member
Joined: 25-Jul-2003
Posts: 4286
From: Unknown

@Petah

Quote:

Petah wrote:
One explanation would be that the "developers" of AmigaOS software in 2012 tend to favor AMOS-style hack-languages like "Hollywood" over proper C. After all, reading tutorials is tiresome and doesn't allow for quick results. Small, tight binaries with low stack size requirements, proper Amiga Style Guide UI compliancy and pretty, custom made icons don't seem to be of much value this spring.

Bull.. A lot of software on linux also uses wget simply because it is convient and already there.
Why go through the trouble of coding something twice when you can just use wget which works very well for this purpose?

I really dont understand why people would complain about this.


But i guess you rather want the little developing resources we have be wasted on this instead?

Last edited by Tomas on 18-Feb-2012 at 06:22 PM.

 Status: Offline
Profile     Report this post  
CodeSmith 
Re: What's with all these program needing WGet?
Posted on 19-Feb-2012 2:48:53
#29 ]
Elite Member
Joined: 8-Mar-2003
Posts: 3045
From: USA

@Tomas

I think the problem people have with wget on the amiga is that it's not part of the OS like it is on Linux. If I use Ubuntu I can freely use wget on any code I write, because I know that it's very likely that it's already there for me. On the amiga, it's Yet Another F'ing Thing People Need To Install if they want to run my program. And heaven help the poor end user if my program depends on version A of wget (eg because it uses some commandline parameter) and some other program depends on version B. It's a special case of DLL Hell.

 Status: Offline
Profile     Report this post  
salass00 
Re: What's with all these program needing WGet?
Posted on 19-Feb-2012 8:17:35
#30 ]
Elite Member
Joined: 31-Oct-2003
Posts: 2707
From: Finland

I'm surprised no-one else has mentioned it yet, but there is also Chris Young's excellent http-handler:
http://aminet.net/package/comm/www/httphandler

Would be nice if it was included with the OS IMHO.

Edit: Just saw there is already an OpenAmiga project for it:
http://openamiga.org/?function=viewproject&projectid=63

Last edited by salass00 on 19-Feb-2012 at 08:29 AM.
Last edited by salass00 on 19-Feb-2012 at 08:17 AM.

 Status: Offline
Profile     Report this post  
olsen 
Re: What's with all these program needing WGet?
Posted on 19-Feb-2012 9:24:56
#31 ]
Cult Member
Joined: 15-Aug-2004
Posts: 774
From: Germany

@Hypex

Quote:

Hypex wrote:
I'm seeing a number of programs that use WGet. And I wonder why they are programmed to need an external CLI command just to load some data off the Internet?


Because it often results in more robust software and sometimes even is a time-saver?

Quote:

Executable binaries shouldn't be requiring a 3rd party tool for this kind of thing. Libraries should be used for this. For instance OS4 has the Roadshow API and also a BSDSocket API like with other Amiga Internet stacks. THat would be the proper way to do it.


I used to believe in that, too. There's nothing like learning how something works by implementing it all by yourself. Once you have earned this knowledge, you can fall back onto "middleware" such as the "wget" command, if it does the job for you. And because you know how to pull off the task all by yourself, should the middleware let you down, you can either fix it or find a different solution.

I still write software mostly using this approach. Once you understand the problem, you will be able to judge better which solution for the problem is most appropriate. But in some cases this judgement means that you won't use your own home-grown solution, but rather somebody else's that is readily available.

For example, given the choice between writing my own HTTP code from scratch or reusing code from "curl" or "wget", I would look into reusing existing code first. The primary reason here would be that reusable code may be more stable than something I would have to implement and test all by myself. This usually makes for more robust software and you usually don't lose too much time, compared to writing the code all by yourself. Sometimes you can even save time by doing so.

In the case of HTTP, I already wrote my own HTTP 1.1 client/server code more than a decade ago, and I had the devil of a time getting it right. The problem was that the HTTP 1.1 specification does not state in great detail how chunked data was supposed to be handled. You could get it subtly wrong, and the code would look right, but some clients and some servers would give you grief. Of course, my own client/server code always worked perfectly together, since the bugs complemented each other.

Quote:

Okay so that would require some learning. But if it is only a simple TCP transaction then the next best thing is a DOS handler. OS4 provides a TCP: device just like Miami did. Why isn't this being used more often?


Because you still have to get the protocol underlying the respective service right? These days this is not a trivial matter. Few protocols are as simplistic as HTTP 1.0 is.

For example, years ago I wrote an SMTP proxy in 'C' which performs spam detection as e-mail passes through it. As part of the task, the proxy not only has to implement the SMTP protocol robustly, but it also has to robustly process the e-mail message contents, which is surprisingly complex. Surprisingly complex meaning "if I had known what I was getting myself into, I would not even have started this project".

 Status: Offline
Profile     Report this post  
Kicko 
Re: What's with all these program needing WGet?
Posted on 19-Feb-2012 9:29:34
#32 ]
Elite Member
Joined: 19-Jun-2004
Posts: 5009
From: Sweden

To get loose of all this dependencies on 3rd party tools it would be good to make such tools/libs/handlers come with the OS. I mean stuff that many apps are dependent on. speciall amiga version, not ports.

 Status: Offline
Profile     Report this post  
opi 
Re: What's with all these program needing WGet?
Posted on 19-Feb-2012 11:07:38
#33 ]
Team Member
Joined: 2-Mar-2005
Posts: 2752
From: Poland

@CodeSmith

Quote:
is that it's not part of the OS like it is on Linux


Well, it's not part of Linux per se, but I'm splitting hair here. I just want to point out why it's OK to depend on wget under Linux distribution: Linux packages do resolve dependency tree. If your software requires wget, you'll get wget installed. Since Amiga does not have that you need to relay on user to have it or stick it into your archive and satisfy GNU licence requiremenets.

_________________
OpenWindows Initiative. Port PS3 hardware to bananas. For free. Join today and receive expired $50 cupon from AI!

 Status: Offline
Profile     Report this post  
Arko 
Re: What's with all these program needing WGet?
Posted on 19-Feb-2012 13:08:29
#34 ]
Super Member
Joined: 17-Jan-2007
Posts: 1989
From: Unknown

@Hypex

Quote:

I'm seeing a number of programs that use WGet. And I wonder why they are programmed to need an external CLI command just to load some data off the Internet?


Because nothing like this is available under C:

Quote:

For instance OS4 has the Roadshow API and also a BSDSocket API like with other Amiga Internet stacks.


The internet stack is a level under the application, and wget is application layer program, they are not part of the internet stack:

http://en.wikipedia.org/wiki/Application_Layer


Quote:

So, after all that, why aren't OS4 programs using the TCP: device? It's not too hard. Do it! :-


You are asking every program inluding its own wget implementation, most programmers are glad if they can use something ready.

_________________
AmigaONE. Haha. Just because you can put label on it does not make it Amiga.

I borrowed this comments from here (#27 & #28):
http://amigaworld.net/modules/newbb/viewtopic.php?topic_id=38873&forum=2&start=20&order=0

 Status: Offline
Profile     Report this post  
djrikki 
Re: What's with all these program needing WGet?
Posted on 19-Feb-2012 14:14:30
#35 ]
Elite Member
Joined: 22-Jun-2010
Posts: 2077
From: Grimsby, UK

I too share the opinion of Olsen who explains all very clearly. Why reinvent the wheel? If there are tried-and-tested tools out there already then no-one should be afraid to use them in their own developments.

Particularly in regard to applications made in Hollywood of which several already use Wget or Curl in one way or another. Andreas, the developer of Hollywood, plans to add network support (or has already started) to the core language.
Naturally this is additional work on his part to allow developers to more closely develop this kind of functionality into their project. In the meantime developers (irrelevant of the programming language) are free to use tried-and-tested alternatives that satisfy a requirement without impunity.

EDIT: lol a news item has just this moment appeared stating this... talk about timing.

Last edited by djrikki on 19-Feb-2012 at 02:18 PM.

_________________

 Status: Offline
Profile     Report this post  
vidarh 
Re: What's with all these program needing WGet?
Posted on 20-Feb-2012 11:05:21
#36 ]
Cult Member
Joined: 4-Jan-2010
Posts: 580
From: London, UK (ex-pat; originally from Norway)

@Hypex

Quote:
On Linux WGet would seem to be unnecessary to me as mostly everything is opened as a file and I'd expect a URL to be handled as such without any work.


It can be, with right FUSE filesystems installed, but next to noone has those installed, and almost everyone has wget.

Part of the problem with using a filesystem handler for that is that there's so many options that can be provided that duplicating the functionality of a tool like wget or curl (the primary developer of Curl is an ex-Amigan by the way) with the limited API of the filesystem operations is really hard to do without making it extremely messy.

And in this case, 99% of the time the network is what is going to limit performance anyway.

_________________
Wiki for new/returning Amiga users - Projects: ACE basic compiler / FrexxEd / Git

 Status: Offline
Profile     Report this post  
Hypex 
Re: What's with all these program needing WGet?
Posted on 20-Feb-2012 14:01:16
#37 ]
Elite Member
Joined: 6-May-2007
Posts: 11180
From: Greensborough, Australia

@ChaosLord

That would be better yes. Then all we need is an already running process that runs the code which can be contacted for services.

 Status: Offline
Profile     Report this post  
Fab 
Re: What's with all these program needing WGet?
Posted on 20-Feb-2012 15:45:10
#38 ]
Super Member
Joined: 17-Mar-2004
Posts: 1178
From: Unknown

I don't think calling an external command from a program is very elegant, actually (especially if the said command not integrated in the OS and that various versions are available). For scripts, it's another matter, of course, nothing wrong there.

For regular programs, i'd rather use something like libcurl (it supports most of the HTTP protocol properly and many other protocols as well). Its API is quite straightforward, and downloading something only takes a couple lines of code (and it's obviously much more complete than doing some "type TCP:blah/80" which totally ignores the HTTP protocol).

And regarding the reusability concern, i think a shared library of libcurl is the way to go (not some .so monstruosity of course).

 Status: Offline
Profile     Report this post  
paolone 
Re: What's with all these program needing WGet?
Posted on 20-Feb-2012 16:05:48
#39 ]
Super Member
Joined: 24-Sep-2007
Posts: 1143
From: Unknown

@Tomas

Quote:
I really dont understand why people would complain about this.


The Amiga way: reinventing the wheel, everytime.

I think anywhere else people would complain if software needs other software to run, it always happened since the day someone invented libraries and interpreters.

 Status: Offline
Profile     Report this post  
Tomas 
Re: What's with all these program needing WGet?
Posted on 20-Feb-2012 16:15:34
#40 ]
Elite Member
Joined: 25-Jul-2003
Posts: 4286
From: Unknown

@CodeSmith

Quote:

CodeSmith wrote:
@Tomas

I think the problem people have with wget on the amiga is that it's not part of the OS like it is on Linux. If I use Ubuntu I can freely use wget on any code I write, because I know that it's very likely that it's already there for me. On the amiga, it's Yet Another F'ing Thing People Need To Install if they want to run my program. And heaven help the poor end user if my program depends on version A of wget (eg because it uses some commandline parameter) and some other program depends on version B. It's a special case of DLL Hell.

And i recall having to use apt-get for it back when i used debian.
And i still dont see the point since this is pretty much same with any app/game these days. There is nearly always some kind of library or something that it depends on.

Much easier to download wget than trying to find some library that might not even be documented in the readme/manual.

 Status: Offline
Profile     Report this post  
Goto page ( Previous Page 1 | 2 | 3 Next Page )

[ home ][ about us ][ privacy ] [ forums ][ classifieds ] [ links ][ news archive ] [ link to us ][ user account ]
Copyright (C) 2000 - 2019 Amigaworld.net.
Amigaworld.net was originally founded by David Doyle