Ok, this is making me crazy. I have been working on this for months. I can write the contents of a web page to a file... however if I try and hit on a web site that is a virtual server it won't work. For example my web server defaults to "woibbs.org/index.shtml" when I try to grab "portzero.net/index.shtml". maybe someone can tell me how I can over come this problem, cus I can't figure it out. Below is a link to source code I have written already:

http://woibbs.org/afa.c

once compiled it would take an argment like this:

afa portzero.net/index.shtml ouput_file.txt

Drew

Last edited by pikadroo on 23-Jun-2005 at 12:03 AM.
Last edited by pikadroo on 23-Jun-2005 at 12:02 AM.

@pikadroo

A few notes:

1) On line 72, use "sock.i.sin_port=htons(80);" instead of "sock.i.sin_port=80;" Since
the code mentions Solaris I assume you are running this in a SPARC but this will break
on e.g. x86.

2) If you are only fetching a single page it is probably easier to use HTTP/1.0 instead
of HTTP/1.1.

3) On line 86, the 4:th argument to sprintf, host, should probably be hostname.

-Tesla aka. Archprogrammer

@Tesla

Ya know what? I dunno why I even bother asking anyplace else. Amigaworld always seems to have the answers I need. Christ... Tesla... you RULE! THANKS A BUNCH!


Those are all for you!

Drew

Last edited by pikadroo on 23-Jun-2005 at 01:24 AM.

@Tesla

Whoops... maybe I spoke too soon. Although I have it working on OS X and FreeBSD I now need it to work on the Amiga One os4. =(

Any more nuggets of thought?

Drew

@pikadroo

The Amiga socket API is slightly different from the BSD (Unix) socket API.
Most things work the same there are a few diffrences, e.g.:

struct sockaddr_in contains a sin_length member, so to initialise the address one
would write e.g.:

struct sockaddr_in target;

memset(&target,0,sizeof(target));
target.sin_addr.s_addr=inet_addr("10.200.8.2");
target.sin_port=htons(80);
target.sin_family=AF_INET;
target.sin_length=sizeof(target);

But, if you are using clib2 you can attempt to use the built-in socket support
from there instead. (I seem to recall this was a easy as linking with -lnet, but do
not take my word for it)

Oops. I have run out of time for the moment, but do feel free to ask if there is anything else.

-Tesla aka. Archprogrammer

@Tesla

Quote:

Tesla wrote: struct sockaddr_in target; memset(&target,0,sizeof(target)); target.sin_addr.s_addr=inet_addr("10.200.8.2"); target.sin_port=htons(80); target.sin_family=AF_INET; target.sin_length=sizeof(target);

I don't see how that is much different other then there is no spaces in a few of them. I seem to be having trouble with memset... Damn... Here I will put online the source again with what I have on the A1 so far. Same address as before:

woibbs.org/afa.c

Drew

@pikadroo

No, there are not so many differences, at least not if you use clib2.
If you use the bsdsocket.library API directly, you will notice it a bit more.

To port you program to AmigaOS4:

1) Add the declaration "extern int h_errno;" to your file.
2) Compile with "gcc afa.c -lnet"

All done.

Also, I noticed another bug in your program. main() should be declared as
returning "int" and the return code should be EXIT_SUCCESS (as defined
in stdlib.h)

-Tesla aka. Archprogrammer

@Tesla

Quote:

To port you program to AmigaOS4: 1) Add the declaration "extern int h_errno;" to your file. 2) Compile with "gcc afa.c -lnet"

How do I know if I have clib2 installed? Or is this just part of the OS4 SDK? Also, the declaration you have there... I just use everything inside the quotes I assume?


Quote:

Also, I noticed another bug in your program. main() should be declared as returning "int" and the return code should be EXIT_SUCCESS (as defined in stdlib.h)

Yeah, I am kinda just trying to get it to work before I add all those fancy thingies. =D

I am gonna try this in a bit, if it works I will buy ya a beer. =)

Drew

@Tesla

BAH! IT WORKS! HOLY ####! LOOK AT THAT! WHOOOO HOOO ROCK AND ROLL!!!!

Anyhow... What would be your beer of choice? it's time to celebrate. I could go to a Guinness.

I can't thank you enough for the help, I will be sure to add your name in the credits =D

Drew

Last edited by pikadroo on 28-Jun-2005 at 06:21 PM.

@pikadroo

Quote:

BAH! IT WORKS! HOLY ####! LOOK AT THAT! WHOOOO HOOO ROCK AND ROLL!!!!

Hooray!

Quote:

Anyhow... What would be your beer of choice? it's time to celebrate. I could go to a Guinness.

Well, actually... I do not like beer. I mostly drink tea and perhaps a wee amount of cider now and then.
But thanks for the offer anyway.

Also, since I am a bit curious - what are you intending to do with this code? I already have working
portable HTTP code in C if you want a slightly higher-level API.
(Tested on AmigaOS4, x86 Slackware GNU/Linux, 32- and 64-bit SPARC Solaris)

-Tesla aka. Archprogrammer

@Tesla

Quote:

Tesla wrote: Well, actually... I do not like beer. I mostly drink tea and perhaps a wee amount of cider now and then.

Hummm, tea sounds good tell ya the truth. Will that be one lump or two?

Quote:

Also, since I am a bit curious - what are you intending to do with this code? I already have working portable HTTP code in C if you want a slightly higher-level API.

Is it online someplace or would you have to email it to me? I would like to take a look at it, but for this project simple is better. The application is for a real time BBS listing site. Every 5 minutes it hits a page with a password and user name to update that users BBS listing. Of course since the site it's self is built on a Windows NT Web Server the only update program they provide is a Windows version. Even though they have a perl version on the site, installing perl on OS4 shouldn't have to be a requirement to being able to use the site. Doing it this way will make it more appealing to sysops. Cool huh?

Drew

@pikadroo

Quote:

by pikadroo on 29-Jun-2005 19:22:41 Is it online someplace or would you have to email it to me? I would like to take a look at it, but for this project simple is better. The application is for a real time BBS listing site. Every 5 minutes it hits a page with a password and user name to update that users BBS listing. Of course since the site it's self is built on a Windows NT Web Server the only update program they provide is a Windows version. Even though they have a perl version on the site, installing perl on OS4 shouldn't have to be a requirement to being able to use the site. Doing it this way will make it more appealing to sysops. Cool huh?

I have put it online at: http://och.nu/tesla/htlib.tar.gz

The archive contains the http code (overkill if _all_ you want to do is to fetch an URL) and
a small example program similar to your afa.c (except it takes a complete URL).

Of special interest to you might be that the library supports passwords (only simple auth at
the moment, no digest auth yet) if you present it with an URL like e.g.:

http://username:password@some.server.net/path/file.html

It also supports alternate ports (e.g. some.server.net:3000) and you have the source
to look at.

-Tesla aka. Archprogrammer

@Tesla

Now I seem to be having trouble with something I thought I added into this thing. If the site is down (ie does not exist) I get a seg fault and a pretty little core file. any thoughts? Wouldn't be an issue but believe it or not a site based on listing only currently online BBSes goes down it's self quite often. LOL

Drew

@pikadroo

Sorry about the late reply. I have been a bit busy as of late.

Quote:

Now I seem to be having trouble with something I thought I added into this thing. If the site is down (ie does not exist) I get a seg fault and a pretty little core file. any thoughts?

Yes. This is because after the call to gethostbyname() you have a memcpy() call before
you check if the gethostbyname() call succeeded. Move this to after the if-statement
and it should be ok.

You could easily have caught this yourself by using a debugger such as gdb or dbx (Solaris).

I am not quite sure how much programming experience you have (I would guess less than 2
years with C), but investing time in learing how to use a debugger can be a real timesaver
in the long run.

If you are testing on x86 GNU/Linux, I can also recomment Valgrind.

Oh, and when running your program through a debugger, do not forget to compile with debug
information (option "-g" for gcc and SunPRO C both).

As an example - support you ran your program through the debugger:

----8<----8<----- Do not cut here, you might damage your monitor----------
> gcc afa.c -o afa -g
> gdb afa
GNU gdb 5.3
Copyright 2002 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i386-lembas-linux"...
(gdb) run nowhere.nil outfile
Starting program: /tmp/afa nowhere.nil outfile

Program received signal SIGSEGV, Segmentation fault.
0x080488d4 in main (argc=3, argv=0xbffff7f4) at afa.c:78
78 memcpy(&(sock.i.sin_addr.s_addr),*(hp->h_addr_list),sizeof(struct in_addr));
(gdb)
----8<----8<----- Do not cut here, you might damage your monitor----------

As you can see, this simple error is immediately caught by the debugger
and you even get the line number.


Home this helps.

-Tesla aka. Archprogrammer

@Tesla

Yup, your right 2 years experience with C back about 5 years ago and let me tell you. I didn't do any of this fancy socket stuff. I did programing for a text pager place here. Until a Y2K issue it was a great job.

Funny how it never occured to the owner that a PDP11 wouldn't be Y2K compliant.

Drew

@Tesla

Quote:

Tesla wrote: Sorry about the late reply. I have been a bit busy as of late.

Naw don't worry about it... whenever you get a chance to check in. Ya know I might be going about the internal timer part of the program all wrong. At it's most basic level this program should "sleep(300);" or wait 5 minutes and then hit the site again. Everytime it does this it displays a line in the CLI telling the user what occured, i.e. Update failed, Update success, DNS error, ect... On this line it also puts the date. In GDB now (which I didn't realize was in the OS 4 SDK) the thing makes maybe about let's say 50 updates with no problem and then gives me:


Program received signal SIGBUS, BUS ERROR,
0x0000f030 in max_month.days.0()


Now I assume this either has something to do with a sleep() problem or something with the part where time is written to a sting. Either way, once this happens... RAM, TCP/IP, TCP/IP Subsystem, cnSMTPd, CNet, telser.99, workbench... I think you get the idea... everything goes to guru hell... and once I get too many I get the orange screen of death at the top of the desktop and at that point the system isn't useable anymore.

It's funny how I solved error checking the connection but I caused a whole new error.

Anyhow, I know sleep() isn't a real standard to making a good cron job but I couldn't think or find an example of anything else. If Amiga OS had a CRON program this whole thing would be solved... hell if perl was standard on OS4 I wouldn't even be typing this right now. CNet does do cron events which is what I am testing right now to see if it works being run every 5 minutes in a 1 time shot.

However, some BBS in the future may not have a cron job so I would like to make sure everyone can use it... any 5 minutes solutions? HA! no pun intended... LOL

Drew

@pikadroo

It looks like an error when writing the timestamp string.
What method are you using? strftime()?


char buf[255];
struct tm *timep;
time_t tim;

tim = time(NULL);
timep = gmtime(&tim);

strftime(buf,255,"%I%p",timep);

printf("Time: %s\n",buf);


-Tesla aka. Archprogrammer

@Tesla

ya know... This is really fun as crazy as it's making me at time. Just gotta know when you walk away from it. This is what all update sections look like in how they handle time:

abfcd_update()
{
   struct tm *ptr;
   time_t tm;
   char str[60];

       if(null_mode == 1){
                tm = time(NULL);
                ptr = localtime(&tm);
                strftime(str ,100 , "%A, %B %d, %Y @ %I:%M%p",ptr);

      if(silent_mode == 0){
         printf("abfcd update success: listing refreshed %s\n",str);
      }
      exit(1);
        }


   if(silent_mode == 0){
      tm = time(NULL);
      ptr = localtime(&tm);
      strftime(str ,100 , "%A, %B %d, %Y @ %I:%M%p",ptr);
      printf("abfcd update success: listing refreshed %s\n",str);
        }

abfcd_time();
}

@Tesla

Ok, I think the time stamp issue has been cleared up. That was my bad with the buffer I suspect.


There also seems to be some issue here too:

       
          struct  hostent host,*hp;

/* some other code thingies here */

          hp = gethostbyname(hostname);

@pikadroo

Quote:

Ok, I think the time stamp issue has been cleared up. That was my bad with the buffer I suspect.

Probably, yes. The value (100) you supplied to strftime() told it that the buffer was
100 bytes long while you actually had a buffer which was 60 bytes long.

Quote:

and line 207 is that "hp = gethostbyname(hostname)" line... Now I get what's going on...

The warning about the assignment is probably due to a missing prototype for
gethostbyname(). The prototype usually resides in netdb.h, but it might be
somewhere else in the Amiga header files. (I can not check right now, but
I would guess at unistd.h or so (easy enough to check with grep))

Oh, and for the credits you can just write:
Peter "Tesla" Bengtsson
(Yay! I have another footnote in history )

-Tesla aka. Archprogrammer