I've held on to FORTRAN because of its ability to explicitly define input (and output) formats -- this is a must for extracting data from a number of publicly available datasets.

I'm now trying to convert one of my most useful programs to PASCAL, but I am hung up on how to read multiple variables from a single line.

For example, here's a FORTRAN format I need to emulate:

FORMAT(I3,I7,2X,A25,1X,F6.2,1X,F7.2,3(1X,I4),1X,F4.1,1X,I1)

On a single line, I begin with two integer variables, skip two spaces, one string variable, one space followed by a real variable, skip a space, etc.

I know of examples that involve treating each line as a character array, but I cannot find them now. I also know how to specify write formats, but have seen no examples of applying that to reading lines.

Thanks for any suggestions,

Dave

If values are separated by spaces, you can just use Readln. If you really need to check exact format
(i.e. error out if there are, for example, three spaces instead of two), you can re-format the values read and compare result with the original string.

The values aren't always separated by spaces. These are huge files with sometimes tens of thousands of lines of data. The only way I can effectively search and extract relevant data form them is to find a way to specify that characters 1 through 3 are a country code, characters 13 through 37 are site names, etc.

Dave

PLEASE to do not top post.
You are out of luck then.
A small help may be to declare a record:
type
TInputLine = packed record

    CountryCode: array [1..3] of char;
    FloatValue: array [1..10] of char;
    Separator1: array [1..2] of char;
    ...
    Eoln: array [1..2] of char;
  end;

and then read you file as "file of TInputLine".
This way, you will not need to split the line manually, only to convert numeric fields from strings.

I don't mean to offend, but I find top posting a more efficient way to keep up with what's new in a conversation. I understand old Usenet-users prefer bottom-posting, but I've spent most of my life working in a far older tradition of the news business. In that tradition it never pays to bury the lede.

Dave

Alexander Klenin wrote

PLEASE to do not top post.

On 8/5/2011 12:58 AM, Alexander Klenin wrote: > > PLEASE to do not top post.

It is quite understandable that you have your own preferences. Just note that your chances of getting help are increased if you communicate in the manner preferable to potential helpers ;-)

You may really use records as Alexander suggested or try RegExp's

Bottom-posting has many benefits.

1. Your reply is related to a specific (quoted) context, so it is easy to relate to your reply, and helps the conversation flow.

2. People with disabilities might use Screen Readers. It is much more beneficial for them to have bottom posting because they can then hear what your reply relates to.

3. ... use Google to find the rest. This topic has been covered to death.

I think your major issue is the old time gripe I have too. It's not bottom or top posting, but rather people that are to f*cken lazy to delete quotes! You then get message that contain some 70+ lines of quotes and a one or two line reply at the end. THIS is what annoys me the most. I have to page-down a few time, just to see the reply. Also emails client that can show you the summary of new message often fail to work due to so many quoted lines.

So PLEASE people, don't be so f*cken lazy. Delete unnecessary quotes form replies! Some email clients are even clever enough to only quote what you selected before you clicked on the Reply button.

So this is some kind of address database ?
Any real-world example you could provide with different field separators ? (data may be dummy, of course)

L.

And while we are at it David, PLEASE do not reply to an existing message, and then change the subject line to start a new message thread.

Most email clients use the message header information to generate a hierarchy of replies. So your method seriously screws with message threading!! Unfortunately many here seem to be doing such stupid things.

Regards,
- Graeme -

2011/8/5 Graeme Geldenhuys :

I'd say that in this case the threading is actually correct, since his message _is_ reply to the parent, even if only to the part of it. It is quite different from the case where users just hit "reply" on random message instead of creating a new one.

With respect, formatting text in a way that doesn't conform to the expectation of its readers is a sure way of getting it ignored.

Alexander Klenin wrote

I'd say that in this case the threading is actually correct,

Fair enough, your original reply was directed at Alexander's reply.

Regards,
- Graeme -

A few months ago I was reading lots of SNMP MIBs (sming format), and wrote a parser using a Meta-2 derivative. However in the general case, and particularly for line-oriented input, it's probably a job for either regexes or an (as yet unwritten) equivalent to format() etc. However, error recovery (when a line departs from the accepted format part-way through) is not going to be easy.

What scripting languages can be embedded in FPC/Lazarus these days? If an embedded language was called for each line would the overhead be excessive?

David M. Lawrence wrote

I don't mean to offend,...

And pleas don't add many lines of unnecessary stuff at the end of your messages, bloating the forum.

-Michael

That's one of the most why maillists must have been already dead many years ago. In XXI, it's time for DHTML, AJAX and other cool and useful stuff. IMHO.

I don't think so (IMHO). The advantage with mailing lists is that if I've missed some days I can easily reread the messages without having a continous internet connection (thus I can read on the train, etc), because the mails are stored in my mail client (only once a connection for downloading the mails needed).

Regards,
Sven

David M. Lawrence wrote

I'm now trying to convert one of my most useful programs to PASCAL, but I am hung up on how to read multiple variables from a single line.

A convenient and very versatile way is to do a TStringList.LoadFromFile and loop through the strings.

For each string you can use TStringList.DelimitedText to get the values separated by blanks

    sltokens := TStringList.Create;
    sltokens.DelimitedText:= ' ';
    sltokens.CommaText:= slfile[i];

-Michael

It is quite understandable that you have your own preferences. Just note that your chances of getting help are increased if you communicate in the manner preferable to potential helpers ;-)

Right. When I got a message with only quoted material on the first (screen) page, I kill it immediately.

DoDi

David did not say it explicitly, but I guess the problem is that some fields may not have separators at all, and parsing is based on the fixed field widths. Hence my suggestion of using file of packed records.

Yep, of course you are right.
Thanks,
-Michael

RSS.
Maillist requires you to receive all of the mails, and you must clear your mailbox from all these messages; multiple quoting of one message is trouble, and quoting multiple messages inside one message is absolute hell.

To the topic: in fact, it's quite simple to create such a parser yourself. Alternatively, you might create obj file with a wrapper over C function scanf which does the trick (but this is quite weird way :) ).

Another approach:

Function eatchars(var s:string; i:integer):string;
Begin
Result:=copy(s,1,i);
Delete(s,1,i);
End;

...

Readln(line);
i1:=StrToInt(eatchars(line,3));  //I3
i2:=StrToInt(eatchars(line,7));  //I7
eatchars(line,2);                //X2
s1:=eatchars(line,25);           //A25
eatchars(line,1);                //X1
r1:=StrToFloat(eatchars(line,7); //F6.2
eatchars(line,1);                //X1
r2:=StrToFloat(eatchars(line,8); //F7.2
eatchars(line,1);                //X1
i3:=StrToInt(eatchars(line,4);   //I4
etc...

Ludo

This IMO should read:
      sltokens.Delimiter:= ' ';
      sltokens.DelimitedText:= slfile[i];

DoDi

Fr0sT wrote

ago. In XXI, it's time for DHTML, AJAX and other cool and useful stuff. IMHO.

Nope, I just wish for a NNTP (news group) setup and plain text messages. NNTP was purposely designed for multi-user discussions. It works perfectly for that.

@Sven
And yes, it too can be used in "offline" mode. Simply set your NNTP client to download message locally. And XanaNews (which I believe is an excelent news client) is even written in Delphi. :-)

Regards,
- Graeme -

I don't clear my mailbox. ^^ I'm using GoogleMail for my mailing lists and access that using IMAP. Then I've also added filters that automatically sort the messages into the correct folders and presto. Wunderful. Also if I cancel reading on one computer (my mobile computer) and continue reading on my other one (my home computer) I can continue where I left of if I've resynced the first computer with GoogleMail again. For RSS you need either to use an online service that you can access from different computers or you need some additional software that can manage the state of your read entries. That is why I have my RSS feeds only on one computer.

But enough of that. That's my opinion and personal taste.

Regards,
Sven

I'm aware of that. I've already added your two fpGUI groups around a year ago to my client (I use Thunderbird for that).

Regards,
Sven

The problem with NNTP is that servers such as INN are likely to filter and reject messages they don't like the look of- excessive quoting and so on.

I'm using INN v2 as a storage backend for MLs (and much else), and there's a constant dribble of messages that I don't get to see as a result.

Mark Morgan Lloyd wrote

The problem with NNTP is that servers such as INN are likely to filter and reject messages they don't like the look of- excessive quoting and so on.

Change the filtering of INN, or use a better NNTP server. I have no problems or such filtering issues with the NNTP server I maintain.

Anyway, we are getting way off-topic here. So lets call this part of the discussion done.

Regards,
- Graeme -

sscanf's like constructs are also a possibility. ARRAY OF CONST is strong enough to implement it.

What scripting languages can be embedded in FPC/Lazarus these days? If an embedded language was called for each line would the overhead be excessive?

For this you need a simple interpreter statemachine. For fixed width values, even regex is overkill.

Really? Can values passed through "array of const" be modified?

Alexander Klenin wrote

Really? Can values passed through "array of const" be modified?

No problem, function might return an array of Variants. Or it might accept it as parameter. "array of const" is an array of TVarRec anyway.

And there seems to be already a starting point:
http://www.freepascal.org/docs-html/rtl/sysutils/sscanf.html

Also someone already has implemented a scanf port for Delphi that's based on "array of const" though the license isn't suitable for FPC: http://reocities.com/siliconvalley/grid/9828/Delphi/scanf_d.html

Regards,
Sven

It is based on "array of pointer", not "array of const". I have no doubts that the former is possible -- my question was about the latter.
Anyway, good find -- original poster might find this procedure useable.

Also someone already has implemented a scanf port for Delphi that's based on "array of const" though the license isn't suitable for FPC: http://reocities.com/siliconvalley/grid/9828/Delphi/scanf_d.html

This link doesnot work for me.

sscanf's like constructs are also a possibility. ARRAY OF CONST is strong enough to implement it.

The Fortran output is fixed format. In
FORMAT(I3,I7,2X,A25,1X,F6.2,1X,F7.2,3(1X,I4),1X,F4.1,1X,I1) the first 2 integers can be linked together. 123 followed by 1234567 result in
1231234567. sscanf can't deal with this AFAIK.

Ludo

Lukasz Sokol wrote

So this is some kind of address database ? Any real-world example you could provide with different field separators ? (data may be dummy, of course) L.

Here's a few examples. These lines are from a precipitation station inventory list:
1066054500 LAGHOUAT 33.77 2.93 -999 1878 1989 16.3 0 1066055001 EL ALIOD 32.90 .50 -999 1909 1961 35.5 0 1066055500 TOUGGOURT 33.12 6.13 -999 1908 1990 34.1 0 1066055900 EL OUED 33.50 6.12 -999 1904 1990 11.4 0 1066055902 EL ARFIANE 33.60 5.90 -999 1935 1966 .0 0

The first ten characters are actually two variables: a three-digit country code and a seven-digit station ID code. It is followed by station names, which may or may not have spaces in them. The full name field is 25 characters long. The remaining variables are latitude, longitude, elevation (missing in these cases), first year of record, last year of record, percent missing values, and some descriptor code.

The next example is from a file listing data flags:

10660680001967110001100011000110001100011000110001100011000110001100011000 10660680001968110001100011000110001100011000110001100011000110001100011000 10660680001969110001100011000110001100011000110001100011000110001100011000 10660680001970110001100011000110001100011000110001100011000110001100011000 10660680001971110001100011000110001100011000110001100011000110001100011000

The variables are a three-digit country code, seven-digit station code, four-digit year, and 12 five-digit flag codes.

Now here's an actual data file (in this case, sea-level pressure):

10260096001958101601016010160101601016010170101401013010140101601016010190 10260096001959101801020010170101601015010160101401012010140101601017010200 10260096001960101901016010160101501017010160101401013010140101601017010190 102600960019611021010200-99991015010150-9999-9999-9999-9999-9999-9999-9999

The variables are three-digit country code, seven-digit station code, four digit year, and 12 five-digit pressure measurements (-9999 means missing data).

Last night I tried the record idea suggested by Klenin and couldn't get it to work. I kept getting "can't read or write variables of this type" errors. It may be because I could not figure out how to read the data as a file as a "type of TInputLine."

Anyway, this is the problem. Now I need to catch up on the follow-up posts and see if I can find something that works.

Dave

{$mode objfpc}
type
  TInputLine = packed record
    FCountryCode: array [1..3] of Char;
    FStationCode: array [1..7] of Char;
    FYear: array [1..4] of Char;
    FPressure: array [1..12, 1..5] of Char;
    FEoln: array [1..2] of Char;
  end;
var
  v: TInputLine;
  f: file of TInputLine;
begin
  AssignFile(f, 'input.txt'); Reset(f);
  while not Eof(f) do begin
    Read(f, v);
    Writeln(v.FCountryCode, '#', v.FPressure[12]);
  end;
end.

That's why I said "starting point" ^^

Anyway, good find -- original poster might find this procedure useable.

Might not as you can't use length and precision declarators as with Format. And especially the first one would be what the original poster needs.

Huh O.o It works here. Can somebody else that link, please?

Alternatively you can try this link:
http://www.torry.net/authorsmore.php?id=1441

This seems to be a newer version (1.7 instead of 1.0), but the license isn't mentioned anywhere there... (I haven't looked inside the download though)

Regards,
Sven

While it seems to be exactly what the poster needs it is a solution that's implemented using assembler...

And another point is the license as mentioned by the ReadMe:

=== quote begin ===

The package is free for private and academic use.
For commercial use, please contact the author.

=== quote end ===

Regards,
Sven

Yes, that's the sort of thing I had in mind. I do little very little C so haven't really memorised even the basic functions.

True, I'd not really focussed on the fact that field widths were predefined. I do even less FORTRAN...

Couldn't a function e.g. FortranCard() take a dynamic array of format specifiers and return a dynamic array of variants? Then

   values := FortranCard(stream, ['I3', 'I7', '2X', 'A25', ...

is simply a case of splitting the input into fixed-length fields, the letters being "noise".

For the record, this solution worked and is the one I've implemented in my code.

Later,

Dave