Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-pascal / 2003-05

Writing a file splitter for very large files.

4 messages · 4 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

A.J. Venter

Hi,

I need to write a method to split very large files (8Gb and up) into chunks which can later be concatenated back into the orriginal.

The chunks would need to CD-Size (e.g. around 630MB's).

How could I handle this ?
Does FPC support opening files that size ?
If I were to open the file, then read and copy through byte by byte, would that work ?

I have some theories, but no conclusive ideas on how to implement this. The files in question are uncompressed tarballs. I tried compressing them but both bzip2 and gzip choked out. This happens whether I use the compressions switches to tar (when creating them) or afterwards on the final file.

Any help, greatly appreciated.

Story of my life: "Semper in excretum, set alta variant" A.J. Venter DireqLearn Linux Guru

Reply by email

James Mills
to A.J. Venter
A.J. Venter wrote 11 lines
Hi, I need to write a method to split very large files (8Gb and up) into chunks which can later be concatenated back into the orriginal. The chunks would need to CD-Size (e.g. around 630MB's). How could I handle this ? Does FPC support opening files that size ? If I were to open the file, then read and copy through byte by byte, would that work ?

I'm not 100% sure on this one, others would probably help you more. However I"m sure you can simply open the file with a standard file handle, and then read byte by byte, reading 630Mbytes then writing this out to another file.

cheers
James

Show quoted text 16 lines
I have some theories, but no conclusive ideas on how to implement this. The files in question are uncompressed tarballs. I tried compressing them but both bzip2 and gzip choked out. This happens whether I use the compressions switches to tar (when creating them) or afterwards on the final file. Any help, greatly appreciated. -- Story of my life: "Semper in excretum, set alta variant" A.J. Venter DireqLearn Linux Guru _______________________________________________ fpc-pascal maillist - http://lists.freepascal.org/mailman/listinfo/fpc-pascal

Reply by email

Michael Weinert
to A.J. Venter: I need to write a method to split very large files (8Gb and up) into chunks which can…

Am Dienstag, 6. Mai 2003 16:51 schrieb A.J. Venter:

Hi,

I need to write a method to split very large files (8Gb and up) into chunks which can later be concatenated back into the orriginal. The chunks would need to CD-Size (e.g. around 630MB's).

under which OS?

under linux => man split

Greets, Michael.

Key fingerprint = 3D66 5A8F 53C0 3AD3 3470 74A6 64AC 55D1 8AFB B436 SysQuadrat Michael Weinert Stuttgart Filderstadt-Plattenhardt Tel.: 0711-9970288 Fax: 5360559 Mobil: 0170-4141273 http://www.linux-firewall.de

Reply by email

Jim Wilson
to A.J. Venter: I need to write a method to split very large files (8Gb and up) into chunks which can…
you wrote 9 lines
I need to write a method to split very large files (8Gb and up) into chunks which can later be concatenated back into the orriginal. The chunks would need to CD-Size (e.g. around 630MB's). How could I handle this ? Does FPC support opening files that size ? If I were to open the file, then read and copy through byte by byte, would that work ?

I had written a program like that back in the DOS days. Basically, I did a bunch of blockread/blockwrite sequences, reading from the original and writing to the "temporary" file.

The old source might still be around, if you're interested. Doubt it would compile with FPC though, because I wrote it a few years ago.

Regards,
Jim

Reply by email