My Amiga4000 is showing signs of its age : the MB battery is already dead, the CPU fan (on a WarpEngine 40Mhz) is sometimes making some unusual noise... Even the HD (UW-SCSI2) is coughting from time to time (not too often, hopefully), like if the head of the drive got stuck or slowed in its moves.
I really hope a AOS4.0 system to appear soon before my A4000 die !

Two days ago, while I was downloading my mails from the POP server, the HD got mad again. I had to click about a thousand times on the several "checksum error on block..." requesters to make it finish its read/write operations before I shut down the machine. After one night rest, my Amiga is up again and the HD is working properly.
But some files in Thor's database got corrupted.

I ran ValidateThor on the database. It took very long, but seemed to work properly (detecting the corrupted files which were stored in a new conference ending in ".old"). I tried to open Thor, but all I got was a loading interface, nothing displayed in the windows, a busy pointer, all in a kind of endless loop. It's as if Thor was trying to read the database (Thor automatically open the "EMail" conference at start up), but there was no disk activity. The CPU was very busy however, everything else was slowed down. As Thor interface did not reply to anything; I had to reboot the machine.

I tried to delete the old conference (the one created by ValidateThor and containing the corrupted messages) using ARexx and BBSread, and ran ValidateThor again. But Thor still cannot open the EMail conference.

I checked the "EMail" conference using Thor's CheckDatabase.br ARexx script, and the conference seems ok.

One thing I noticed is that the conference is configured with a "-1" as "BBSConfNr" (the internal conference number). I think that at least this number should be positive.
Edit: This is normal as the latest version of Thor does not use this tag anymore for numbering the conferences. (See post #3)

What else can I try ?

Last edited by AmigaPhil on 11-Dec-2006 at 11:16 PM.
Last edited by AmigaPhil on 04-Dec-2006 at 05:23 PM.

@AmigaPhil

Be patient about expecting responses from around here about Thor.

My ISP demands authentication which Thor cannot provide so I surrendered and stopped using Thor somewhat over a year ago.

Even back then I could get no help about specific steps one could take to sort out corrupted database conferences. I have many, many such that I would like to recover.

About 2 years ago or so there was a move afoot to open source the Thor code, but nothing came of it.

I would love to have an amiga based usenet reader integrated with my email program.

The closest promising Amiga thing is extending YAM2.5 once it gets to a stable version (could be soon!) to handle usenet. This is theoretically possible because YAM is opensource.

*IF* that should ever happen, next would be more pressure on Thor programmers to release the code so these broken Thor database files could be compared to the proper structures. That is the most straight forward way of doing recovery.

George Wyche

_________________
..effects of civilization upon...nature, the growing gap between what education was supposed to accomplish and what it consisted of, the national debt and...high taxes, the problem of the excess cost of medical care -- Philip Wylie, 1951

@ggw

Quote:

My ISP demands authentication which Thor cannot provide so I surrendered and stopped using Thor somewhat over a year ago.

Connexion to your ISP and sorting/managing your mails can be separated operations. You can download your mails/news using other methods (eg.: using the TCP: device in a AmigaDOS script), and then use Thor to parse the downloaded files and integrate the messages to Thor's database.


Althought Thor is quiet old (not updated since 1999, IIRC) and lack some features, I still use it mainly because of ARexx. Also, Thor was the first stable and powerfull mail/new manager I found when I first got an Internet access. Today, I still have several thousands of messages in the database. Some of the messages I keep contains valuable information like passwords.

I have not yet found a solution to my corrupted database problem, but I keep trying (I made a backup of the corrupted database so that I can experiment without taking the risk to damage it futhermore).
Rem.: The "-1" I found strange as the Internal BBS number seems to be ok with the latest version of Thor (2.6a): the conferences numbers are now taken from their filenames.

I'm currently writing a ARexx script which will rewrite all the conferences and the messages in a BBS system. The script is almost finished, but I'm facing a bug where I got a level 30 error ("Failed to parse arguments") from BBSread on the following line:

'CONFIGCONF bbsname "'bbsn'" confname "'confn'" repstr "'CONFDATA.REPLYSTRING'"'

@AmigaPhil

Assuming that CONFIGCONF is a rexx instruction, if you write it this way:

'CONFIGCONF bbsname "'bbsn'" confname "'confn'" repstr '"'"CONFDATA.REPLYSTRING"'"

then the CONFDATA.REPLYSTRING will be enclosed in quotes, not double-quotes and therefore can contain double-quotes.

Last edited by yak on 20-Nov-2006 at 04:34 PM.

@yak

Quote:

Assuming that CONFIGCONF is a rexx instruction

Yes, CONFIGCONF is a command addressed to BBSread port.

Quote:

then the CONFDATA.REPLYSTRING will be enclosed in quotes, not double-quotes and therefore can contain double-quotes.

I'll try that, thanks. But then... what if the string contain a single quote (something like "I'm out of office") ?

@AmigaPhil

Well, that's a real problem then. AFAIK the parsing of external commands parameters is done by the application (after substituting variables with their values by Rexx). Maybe the app's documentation can give you some hints?

@yak

It seems there is no more active resources for Thor. Thor web site is still there, but in abandonned state. Most developpers have gone to other platforms, I think.

Anyway, from ARexx manual, I read:
Quote:

To include string delimiter characters in the string, use a double-delimiter sequence ('' or"").

So I'll try to check all strings before they are passed as parameters to CONFIGCONF, change " to "", and see if it helps.

Edit: Doubling the string delimiter ("") does not help here, but I found that escaping the double-quote with a "*" (asterix) does the trick. Back to testing the script now...

Last edited by AmigaPhil on 20-Nov-2006 at 08:38 PM.
Last edited by AmigaPhil on 20-Nov-2006 at 08:36 PM.

@AmigaPhil

Quote:

Doubling the string delimiter ("") does not help here, but I found that escaping the double-quote with a "*" (asterix) does the trick.

I still think this is an application specific thing but good to know anyway . Good luck.

@yak

Quote:

I still think this is an application specific thing but good to know anyway

Yes. I probably read something about that in Thor's docs, but cannot find where anymore.

I'm still stuck with my broken database problem. Hopefully, I can access the database with BBSREAD and ARexx, so I keep trying.
Running the ARexx script I wrote (it took half a night to complete) did not help; Thor's interface still "freeze" when trying to read the database. Investigating deeper into the database, I found that some of the messages have an awfully long number string as the CREATIONDATE field: 10 digits for a date expressed in seconds since 1978, that bring the creation date to somewhere in 2028 ! Also, I found some characters which shouldn't be there in the message headers (eg.: no LF are allowed in text fields).

Tonight, I will run my ARexx script again, modified to clean the message headers, and see what happen. I will write here if I find anything that might help other Thor users.

Quote:

I found that some of the messages have an awfully long number string as the CREATIONDATE field: 10 digits for a date expressed in seconds since 1978

BTW, after 9th September 2009, ARexx scripts dealing with Thor that do not care to use "NUMERIC DIGITS 10" will have trouble with dates, as the numbers of seconds since 1978 will have more than the 9 digits ARexx can handle by default.

(My database is still not fixed. I think I need to check deeper into the message body for unsafe characters.)

Last edited by AmigaPhil on 11-Dec-2006 at 11:13 PM.

Quote:

(Thor automatically open the "EMail" conference at start up)

I've discovered that when opening, Thor is looking for a SYSTEM_ENTRY_CONFLIST environment variable; and if found, it pops up a conference list from which you can choose the conference to open.
(Just create ENV:THOR/SYSTEM_ENTRY_CONFLIST with whatever value, e.g.: "1", to enable this feature.)
Now at least I can browse some conferences from within Thor.

I however still haven't found WHAT is making Thor to hang, and in WHICH messages.
Keep trying ...

In my ARexx script I'm tweaking, I have a small routine to replace any sequence of the TAB (ascii 9) character by a single space in the header of a message.

outstr = space(instr,1,d2c(9))
outstr = translate(outstr,' ',d2c(9))

I finally found that it was duplicate messages in a conference that made Thor going mad. Two or more messages were sharing the same MSGID, and that puzzled Thor while trying to build a list of messages. (There were howerver other conferences with duplicate messages that were still readable by Thor. So I guess Thor is using more than the MSGID to identify messages.)

The last run of my script took about 13 hours and 39 minutes to rewrite 24663 messages stored in 148 conferences. But now, Thor is back !

For anyone interested, I've uploaded my script to Aminet: BBSsalvage

Last edited by AmigaPhil on 11-Dec-2006 at 11:47 PM.

@AmigaPhil

Thanks for your work. I'll go have a look see. I really do want my messages back.

ggw

_________________
..effects of civilization upon...nature, the growing gap between what education was supposed to accomplish and what it consisted of, the national debt and...high taxes, the problem of the excess cost of medical care -- Philip Wylie, 1951

@ggw

I've uploaded a new version (V1.1) where the RENAMEBIN option is fully funtional (both the name of a binary in a message AND the name of the file itself will be renamed when needed to something AmigaOS friendly).
Not sure this can help to fix a broken system (e.g.: Thor doesn't need to check a binary file when building up a messages list), but it is convenient anyway for later manipulations of files.

How are you going ? Have you fixed your database problem ?

@AmigaPhil

Thanks to TonyW. I had only about 6000 messages to do. It took this AmigaOne 1 hour 18 minutes.

Indeed, after that, I was able to process all of my thor files without problem.

OK. Now for the *next* thing.

Several years ago I moved my A1200 Thor binary files over to my AmigaOne. Sadly, that was a disaster! Thor refused to recognize anything at all. I presented a stiff upper lip and abandoned them to start anew.

Now, with your experience, what might I do with these binary data files sitting here? How can I induce ANY software to accept any of them?

I don't care what the conferences were called before... call them A through Z (because I don't remember). I want the emails and news items that were inside.

Can you give some hints about reintroducing a miscellaeous BBS data file back into Thor?


George Wyche

_________________
..effects of civilization upon...nature, the growing gap between what education was supposed to accomplish and what it consisted of, the national debt and...high taxes, the problem of the excess cost of medical care -- Philip Wylie, 1951

@ggw

(I've uploaded yet another version to Aminet as I forgot to escape the single quote itself in filename in the previous update)

Quote:

Several years ago I moved my A1200 Thor binary files over to my AmigaOne

You mean just the database, the whole Thor installation (including the executables), or the decoded binary attachments of messages ?

Quote:

Sadly, that was a disaster! Thor refused to recognize anything at all.

(I once too messed the installation completely when I started to use Thor.)
You can move a database (that is the "Data" directory) to another location, but make sure to edit the environment variable THOR/BBSDataPath to the new path. Also, from within Thor, check in the configuration if the paths still point to where they should. Remember that before doing any such modification of Thor installation you have to make sure there's nothing left related to Thor (BBSread) in memory: Re-boot your machine, or at least type the command "avail flush" from shell first.

Let's see if I've understood: You tried to move your Thor installation from your A1200 to your AOne, but it didn't work. So you started with a new installation of Thor on your AOne, and now wish to get back the messages you had on the A1200. Am I right ?

Can Thor still run on your A1200 and do you have normal access to your old messages ?

@AmigaPhil

>Let's see if I've understood: You tried to move your
>Thor installation from your A1200 to your AOne, but
>it didn't work. So you started with a new installation
>of Thor on your AOne, and now wish to get back the
>messages you had on the A1200. Am I right ?
>
>Can Thor still run on your A1200 and do you have normal
>access to your old messages ?

The A1200 is far from operable. You are correct about my wanting the Thor Data. I will try to so as you suggested about placing the following files (which I had backed up) to someplace... like DData (instead of Data) and give Thor another try. I have Christmas, volunteer, and work getting in the way, but I'll report back. These are what I have to work with:

Data/BBS.data
Data/BBS.data.bak
Data/BBS.data.dup
Data/BBS0/.header
Data/BBS0/Conf.data
Data/BBS0/Conf.data.bak
Data/BBS0/Conf.data.dup
Data/BBS0/Event.data
Data/BBS0/Event.tags
-----------------------------------------------------
In general the following have these sequences:
0-2,4,6-8,10,11,14-22,24-30,34-39,41,43,45-47,49-54,82,88,89,92,93,95-114,118-132
However, the following are missing
Data/BBS0/MsgData# 34
Data/BBS0/MsgHash# none missing
Data/BBS0/MsgHead# none missing
Data/BBS0/MsgMarked# 19,24,26,27,34,35,37,38,89,96,118-121,126,127,129-131
Data/BBS0/MsgText# 98
------------------------------------------------------
Data/BBS0/PassConf.data
Data/BBS0/SortMail.cfg
Data/BBS0/SortMail.count
Data/BBS0/TCPExtConfig
Data/BBS0/TCPTimestamp.ng
Data/BBS0/Text1254.msg 2303,2503-2507,2510,2511,2531-2533,3923
Data/BBS0/User.data
Data/BBS0/User.hash
Data/BBS0/User.tags
Data/BBS0/msg
Data/BBS0/tcp_tmp/n.1
Data/BBS0/tcp_tmp/n.progress
Data/BBSType.data
Data/BBSType.data.bak
Data/BBSType.data.dup
Data/Charsets.data
Data/ErrorLog
Data/Globals.data
Data/Globals.data.bak
Data/Globals.data.dup

Should I try to fake the missing ones by putting in empty files for them?

ggw

_________________
..effects of civilization upon...nature, the growing gap between what education was supposed to accomplish and what it consisted of, the national debt and...high taxes, the problem of the excess cost of medical care -- Philip Wylie, 1951

@ggw

Quote:

Should I try to fake the missing ones by putting in empty files for them?

No, this won't help. Either the missing files are from conferences you had deleted, or some of those missing files are lost conferences anyway and there is nothing you can do to get them back.

It would have been easier if your old (A1200) database was still up and running... But you could try to read it on your AOne.

Let's say your NEW (AOne) database is in "Data:Thor/Data/" directory.
Copy your (old) DData directory in Data:Thor/
Reboot (to cleanup mem) and edit ENV:THOR/BBSDataPath to the new path ("Data:Thor/DData/")
Open Thor and see if you can enter.
If not, reboot again (and remember to edit the environment variable again !)
CD to your Thor/bin/ directory and type from shell : "BaseManager VALIDATE ALL"
Run Thor and wait a moment. Hopefully you will be able to browse your old database.

Next step will be to export/import the old database to the new one. We'll see that later.

@AmigaPhil

Well I abandoned work this morning and worked on this. No wonder I'm not rich!

Since I cannot use Thor in its current state with my ISP (because of authentication issues), I saved all my messages from the "current" Thor, then I deleted the whole thing and reinstalled Thor (except non-Thor related OS4 stuff). Then I restored all that stuff I printed in my last msg to this thread.

Did BaseManager VALIDATE ALL (which took 1 second)
Reboot
ValidateTHor internet (which took less than 1 minute)
Thor
--Everything was zero everywhere when it came to messages. No messages seen.

--That was what happened last time, too.

--However, there is all those data files. Many, many megabytes of them.

Sort of at a loss now about these old, desired messages.

ggw

_________________
..effects of civilization upon...nature, the growing gap between what education was supposed to accomplish and what it consisted of, the national debt and...high taxes, the problem of the excess cost of medical care -- Philip Wylie, 1951