I've noticed Michael just added a "wget" package getter to the new packager.

We already had a small discussion about using lNet in the new packaging system and it comes down to this:

The packager needs pure fpc solution, which lNet now is, but won't be later (eg: when openSSL is added)

I'm willing to add the required networking support to the packager if you guys tell me "how". I mean, should I add lNet as a choice to package/base or fcl (not sure on which the packager actualy depends), or just "inside" the packager (eg: not in packages or fcl)?

Personaly, I don't give a damn, I'm going to put lNet in packages/extra later and you need a stripped down version anyhow (eg: you don't need no openSSL support which forces a libc inside).

Please don't start a flamewar about this, all I need to know if you want to use lNet for it, and how should I integrate it with the packager, no politics.

Ales Katona wrote

I've noticed Michael just added a "wget" package getter to the new packager.

Yes.

I know.

I'm willing to add the required networking support to the packager if you guys tell me "how". I mean, should I add lNet as a choice to package/base or fcl (not sure on which the packager actualy depends), or just "inside" the packager (eg: not in packages or fcl)?

As far as I'm concerned, lnet can go in packages/extra/lnet, or better yet, fcl/lnet...

I am already adding lnet and synapse support to fppkg. (to WST too, BTW ;) ) The wget is a fallback and proof of concept implementation.

Personaly, I don't give a damn, I'm going to put lNet in packages/extra later and you need a stripped down version anyhow (eg: you don't need no openSSL support which forces a libc inside).

Proxy support may be nice in the mid-long term...

Please don't start a flamewar about this, all I need to know if you want to use lNet for it, and how should I integrate it with the packager, no politics.

All you need to do is

1. Add lnet to fcl/lnet

2. Add a set of synchronous calls for client use.
I noticed you set it up as asynchronous using an event loop. For command line tools, a synchronous call interface is easier. (I commented on this already to Micha with regard to WST)

2 is not mandatory, it will just make my interface easier.

Michael.

I am already adding lnet and synapse support to fppkg. (to WST too, BTW ;) ) The wget is a fallback and proof of concept implementation.

I've got a skeleton lNet/fppkg done localy now, will finish it tomorrow (and hopefuly find a way to test :) )

I've put lNet to fcl for local setup too but am open to suggestion.

Btw, do you plan to add support for concurrent downloads and "compiling while downloading" later?

Well. Testing should be easy. I committed a test program. Make sure your unit sets the DownloaderClass class variable and set it in the testdownload.pp program uses clause.

After running the test, the FPC homepage should be in fpc.html :-)

I've put lNet to fcl for local setup too but am open to suggestion.

fcl is cool :)

Btw, do you plan to add support for concurrent downloads and "compiling while downloading" later?

Yes, but that will be done using threading. Make sure your objects do not access any global variables. It should be thread-safe.

Michael.

Synapse too? Why not Indy then? :-)

> Personaly, I don't give a damn, I'm going to put lNet in packages/extra > later and you need a stripped down version anyhow (eg: you don't need no > openSSL support which forces a libc inside).

Or a plugin system like used in the rtl for some feats.

Marco van de Voort wrote

Synapse too? Why not Indy then? :-)

yes, indy run on arm-wince :-)

I was planning to add an Indy and ICS implementation as well. Synapse is symply easiest to add. But if you want to do the Indy implementation, please do.

But neither of those will be the default. The default will probably be lnet, as that will be distributed with FPC.

But since I believe the packager should be agnostic with respect to the used TCP/IP suite, I wanted to have many implementations.

Currently committed:
- wget
- synapse
- curlpas

Planned:
- lnet (Ales, will you do this or not, I was actually waiting with my implementation?)
- plain libcurl
- plain curl
- Indy
- ICS
If you want to implement one of Indy/ICS (I know you have experience with them) then I can focus on other things...

The package manager uses a plugin system.
You have to set the DownloaderClass class variable, and then that one will be used.

Michael.

- lnet (Ales, will you do this or not, I was actually waiting with my implementation?)

I mailed you yesterday, I get this error trying to compile fppkg:

Target OS: FreeBSD/ELF for i386
Compiling fprepos.pp
Fatal: Can't find unit streamcoll

Seems it's either not installed with fcl or there's a ppu mismatch so it looks for the source of it, didn't investigate yet (might be related to my lNet makefile changes perhaps).

I do however have most things "done" as far as going without testing can be :)

My "make all install" on freebsd of this morning installed a unit streamcoll, so probably already fixed.

Marco van de Voort wrote

streamcoll My "make all install" on freebsd of this morning installed a unit streamcoll, so probably already fixed.

I confirm, latest 2.1.1 works.

Yes, and I sent you an answer. Didn't you get it ?

Seems it's either not installed with fcl or there's a ppu mismatch so it looks for the source of it, didn't investigate yet (might be related to my lNet makefile changes perhaps).

It's in fcl/inc

I do however have most things "done" as far as going without testing can be :)

Excellent.

Michael.

Ok I tested it and I got an odd bug.

When I "writeln" the buffer contents before writing them to the Dest stream, they are ok. But the resulting file is a garbage of xterm crap (mostly symbols) with exactly same size as the thing I downloaded (completly different data).

I've no idea where to look, since writeln of the buffer looks 100% ok...

Any idea what this might cause? I can commit it so you can test.

Ales

Sound like you are doing something like

write (x,< a number>)

with x a pointer type instead of

write (x[0],< a number >)

IOW writing the pointer value and a random number of bytes because you forgot a dereference.

Oh DOH thanks, it was the pointer stuff. Sometimes I wished fpc warned if you pass a pointer to untyped var.

In any case pkglnet.pas is in now, I tested with the example and it worked. Only HTTP yet, but I'll do FTP ASAP.

Ales

Wanted to ask, how should I process the URL?

Should I also understand ":" as separator for port eg: http://www.shit.com:3030 ?

Shouldn't this be parsed by some common function? Eg:
procedure ParseURL(var Host, URI: string; var Port: string);

This way we can save the hassle for all the backends and ensure same things will get understood.

Ales

This function exists and is used. See uriparser.pp (packages/base/netdb) It's already used in the pkgdownload, pkgsynapse files. (ParseUri function)

This way we can save the hassle for all the backends and ensure same things will get understood.

Yep. As you can see, all has been foreseen. This is the result of many years of work-in-progress ;-)

I have not yet thought about proxy support, I need to think about that.

Michael.