Hi,

I decided to implement the online package manager(opkman) or at least give it a try. The first stable(alpha) version is ready and it works like this: the packages are zipped and stored in a webserver(repository) along with a json file. I chose zip to minimize server load. The json file contains all the necessary info about the packages. Once installed into the IDE, the opkman downloads, then serialize the json file into a package list. The list is displayed in a tree. On request the packages can be downloaded or/and installed. For now the repository is read-only, it contains only eight, well known(I guess?) packages. The nature/type of the eight package is irrelevant for now, since we are only testing the functionality of the opkman. Few notes:

1. Supported versions: Laz 1.6+(FPC 3.0.0+). I did the implementation/testing with Lazarus trunk(FPC 3.0.0)
2. Tested under the following widgetsets: Win32/64, GTK2, Carbon
3. After install, you can find opkman in the "Package" menu
4. Download link:
https://drive.google.com/file/d/0B9Me_c5onmWoVXU3Y1N2dk8xWDQ/view?usp=sharing

Please test! Suggestions are welcome.

PS: Big thanks to @Leledumbo for hosting the repository.

regards,
Balázs
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus-ide.org/pipermail/lazarus/attachments/20161004/8a7f7f9e/attachment.html>

Balázs Székely wrote

I decided to implement the online package manager(opkman) or at least give it a try.

Not to deter from all your efforts, but I think there is already a brilliant solution called "Delphinus". It is an open-source replacement to the ridiculous one included with newer Delphi versions as standard.

All that would be required is to port that code to work with Lazarus IDE (instead of Delphi IDE - or possible work with both IDE's).

So, what's the benefit of Delphinus:
* there is _no_ central repository!
* You can publish new project without notifying the Delphinus author
* It hooks into the Github API, so everything is automated.
* You can have a console app or IDE add-on to install packages.

See the second link for how simple it is to publish a new project.

Delpinus (website and code repository)
https://github.com/Memnarch/Delphinus

Publishing your Project for Delphinus
https://github.com/Memnarch/Delphinus

Regards,
Graeme

Hi Graeme,

Thanks for the feedback! "Delphinus" is indeed an interesting project, let's hope somebody will have the time and energy to convert it to Lazarus. Personally I prefer a central repository, so each package can be checked individually. I'm not a control freak, I just want to make sure that each package in the repository can be compiled/installed.

Balázs

On Tue, Oct 4, 2016 at 3:35 PM, Graeme Geldenhuys via Lazarus <

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus-ide.org/pipermail/lazarus/attachments/20161004/6afd5fcd/attachment.html>

On Tue, Oct 4, 2016 at 9:11 AM, Balázs Székely via Lazarus <

Thanks for sharing it, it's really a great job! :-)

Some suggestions:

* allow the user to use Lazarus-CCR/Github/Bitbucket/GitLab as component server; (it allows anyone to publish their components)
* rename it to something like LazGetIt or LPM (Lazarus package manager) etc., try to send it to be distributed in the Lazarus /components folder, or try to ask Lazarus team about how to improve it, so that it comes installed as default in the official Lazarus IDE; (Lazarus needs a official solution like Delphi's GetIt or Delphinus)
* add the "Uninstall", "Remove", "Download" and "Info" buttons; (Uninstall
- just uninstall it from Lazarus IDE; Remove - uninstall and remove all downloaded files; Download - just download the component sources without install it; Info - show the component README/overview)
* synchronize it to show some already installed component; (it avoid conflict with another installed user component)
* allow it to use HTTPS.

Anyway, I've not seen the sources, I've just installed it on my IDE taking the following awesome window: https://img42.com/6axLU .

[...] * rename it to something like LazGetIt or LPM (Lazarus package manager)

Note: the IDE's package manager is named Lazarus package manager.

Mattias

On Tue, Oct 4, 2016 at 2:02 PM, Mattias Gaertner via Lazarus <

Indeed. :-)

@Silvio

allow the user to use Lazarus-CCR/Github/Bitbucket/GitLab as component

server; (it allows anyone to publish their components) It would take a lot of effort, the project is already complex, although I only completed about 30%. I might add support for Github later, Delphinus is already there and is a great tool.

rename it to something like LazGetIt or LPM (Lazarus package manager) etc.

In my opinion the name is irrelevant for now, it can be renamed later to anything except LazGetit, it sounds like a copy of Delphi's GetIt. It's entirely different project, concept.

add the "Uninstall", "Remove", "Download" and "Info" buttons

"Download To" and "Clean up" buttons are already there. I will add the others soon.

synchronize it to show some already installed component; (it avoid

conflict with another installed user component)
It does show you if the component is installed or not, more over you can see the installed version(the second column, with bold). See attachment.

allow it to use HTTPS

Already supported! You need the install the OpenSSL libraries though.

Thank you for the feedback.

On Tue, Oct 4, 2016 at 7:43 PM, silvioprog via Lazarus <

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus-ide.org/pipermail/lazarus/attachments/20161005/89e70b99/attachment-0001.html> -------------- next part --------------
A non-text attachment was scrubbed...
Name: opkmani.png
Type: image/png
Size: 38383 bytes
Desc: not available
URL: <http://lists.lazarus-ide.org/pipermail/lazarus/attachments/20161005/89e70b99/attachment-0001.png>

The online package manager works well and its operation is smooth thanks to threaded code.

Earlier discussions have recommended fppkg.
http://free-pascal-lazarus.989080.n3.nabble.com/Lazarus-An-online-package-manager-tt4043469.html#none I discussed with GetMem earlier and he says fppkg is not very well suited here. I must take his word on this issue because he is the only person so far to make a functional online package manager. "Lazarus packagemanager" from Darius Blaszyk uses fppkg but does not work. The "Aarre" initiative from Mattias and myself has no functional code either. Several people (including myself) had plans to write such a package manager ... but nobody did until now.

My goal is to include this GetMem's work in Lazarus sources. It also means commit access for him.

Now the code has copies of some FCL classes and a copy of VirtualTreeview. These can be seen as temporary copies. The goal is to integrate management of online package and local packages. The Install/Uninstall Packages window in Lazarus would then show also online packages and this new VirtualTreeview based GUI would show local packages. In essence the new GUI will be an alternative to the old GUI. Competition is good.

What do you people say?

Juha

The info regarding the fppkg manager is not quite correct. There is a testing website for submitting fppkg packages (it's not yet published, though). So why fppkg is not suitable is a bit of a mystery to me.

Michael.

fppkg is perfectly suitable for freepascal packages. Nice clean code. A lazarus package is different(although it has some similarities). Converting between the two package type was too much for me. I quickly gave up.

Balázs

On Fri, Oct 7, 2016 at 4:35 PM, Michael Van Canneyt via Lazarus <

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus-ide.org/pipermail/lazarus/attachments/20161009/f8c48073/attachment.html>

I opened today options of Coolbar of ide (using rt-click on toolbar's grip lines),
and cannot select 2nd toolbar (1st toolbar is selected always with carrot BG color, cant select 2nd).
And cant configure it.
It is bug?
Ubuntu 14.04 gtk2

Alexey
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus-ide.org/pipermail/lazarus/attachments/20161009/b07b9c55/attachment-0001.html> -------------- next part --------------
A non-text attachment was scrubbed...
Name: gcdbnohpjepbehai.png
Type: image/png
Size: 27687 bytes
Desc: not available
URL: <http://lists.lazarus-ide.org/pipermail/lazarus/attachments/20161009/b07b9c55/attachment-0001.png>

On Sun, Oct 9, 2016 at 4:15 AM, Alexey via Lazarus <

You can select it by clicking the empty area at right side of the icons.

It is bug?

Yes, the behavior is counter-intuitive. Clicking the icons should also select it.
If somebody wants to fix it, a patch is welcome.

Juha
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus-ide.org/pipermail/lazarus/attachments/20161009/329419bc/attachment.html>

On Sun, Oct 9, 2016 at 1:50 PM, Juha Manninen via Lazarus <

[...]

Done: http://bugs.freepascal.org/view.php?id=30713 .

Hi,

We have made considerable progress with the online package manager. Next week the central repository will be populated with 30+ packages. Here is a short video(please watch it in 700p).

https://youtu.be/y_hT8u2u5bU

Suggestions are welcome.

Very nice work!

Balázs Székely wrote

Suggestions are welcome.

Just a note: please mind the image policy of Lazarus - see trunk\images\README.txt (if you haven't done it already). Especially please keep track of image sources you use (e.g. in some kind of INFO.txt or whatever) - it's easy when you add images but hard after some years are gone :)

Ondrej

Hi Ondrej,

Thanks for the feedback!
I only download/use free icons(free as a free beer :) ) from sites like this one:
https://www.iconfinder.com/iconsets/32x32-free-design-icons It clearly states: "License: Free for commercial use (Include link to authors website)". I suppose it's ok to use them, if we leave a txt with a link?

Balázs

Balázs Székely wrote

I only download/use free icons(free as a free beer :) )

Be aware that CC license is not "free as a free beer" :)

Exactly. Keep track of used icons and list them by license/author in some text file. See e.g. lcl\images\copyright.txt . It makes things a lot easier in the future. Thanks!

+ Also make sure that you don't list iconfinder.com as a source but the original author - e.g. in case of your link above it's http://www.small-icons.com/packs/32x32-free-design-icons.htm .

Ondrej

Now all that is needed is something like Torry.net which indexes and offers search for all the components. This is something that has kept me using delphi/lazarus instead of Visual C++, as visual C++ has absolutely nothing for websites offering components... I have no clue where msvc users get their components from, they seem to roll their own or use codeproject.com

The current lazarus code repository on sourceforge and spread about throughout the wiki is good, but a website like torry.net that centrally indexes the stuff would be great. In fact I was interested in working on such a project at one time but paid projects were more of a priority so I put it on the back burner.

Also, interesting that Torry still has no real lazarus section AFAIK... it's almost as if torry doesn't much care about fpc/lazarus. There were rumors that they were going to add it at some point; not sure whatever happened.

Online Package Manager shows the packages with "new" icon if the latest release time was in the given number of days, but shows them with normal icon if they are older than that days regardless if they are up-to-date or not.
I my opinion:
- It would be better to show up-to-date packages with normal icon and
- Show all packages with "new" icon if their last release is newer than the installed one...

2016-10-28 08:20 keltezéssel, Balázs Székely via Lazarus írta:

The idea was to notify the user if something has changed in the main repository. With your method non-installed packages wouldn't show up as "new" packages. For example if a completely new package is added to the main repository, you cannot see it as new, which is not optimal in my opinion.Beside if a new update is available a bold NEW text appears in the update column(see attachment).

On Wed, Mar 29, 2017 at 9:59 AM, Péter Gábor via Lazarus <

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus-ide.org/pipermail/lazarus/attachments/20170329/7be09d39/attachment-0001.html> -------------- next part --------------
A non-text attachment was scrubbed...
Name: OPM.png
Type: image/png
Size: 75001 bytes
Desc: not available
URL: <http://lists.lazarus-ide.org/pipermail/lazarus/attachments/20170329/7be09d39/attachment-0001.png>

Hi,

The idea was to notify the user if something has changed in the main repository. With your method non-installed packages wouldn't show up as "new" packages. For example if a completely new package is added to the main repository, you cannot see it as new, which is not optimal in my opinion. Beside if a new update is available a bold NEW text appears in the update column.

Free Pascal - Lazarus mailing list wrote

Of course a newly added package in the repository can/must be marked as new for a period of time.

With your method non-installed packages wouldn't show up as "new" packages.

I was talking about installed packages when I said that "all packages", sorry if it was not clear enough....

> It would be better to show up-to-date packages with normal icon

This means: if I have an installed package and specially if it is up-to-date then it is not new for me (no way), so it does not need my special attention until a the next update... updated packages are not new packages.
Currently I have a package (FPSreadsheet) up-to-date and it is marked as new...

I added a new option(r. 54501): "Show regular icon for newly added packages after install" which will overwrite the current behaviour. By default the option is not checked.
Please test!
<http://free-pascal-lazarus.989080.n3.nabble.com/file/n4051284/OPM.png>

Seems to be working.

2017-03-30 08:09 keltezéssel, Balázs Székely via Lazarus írta:

However, the behaviour of the "on" state of this option is the normal behaviour in general package managers (ie. synaptic) and no option must be changed for it. Who needs to see that a package is new if it is already installed and up-to-date?

Anyway, Thanks!

However, the behaviour of the "on" state of this option is the normal behaviour in general package managers (ie. synaptic) and no option must be changed for it. Who needs to see that a package is new if it is already installed and up-to-date?

The thing is everyone would like a slightly different behavior. Anyway I set the default value is true.

Anyway, Thanks!

You're welcome!

Seems to me you should have at minimum 3 status for packages:

New - Not yet installed but available in repository

Updated - Already installed but a new version is in repository

Up-To-Date - Already installed and installed version is same as in repository

Using "new" for both "not yet installed" and "installed but with available updates" is confusing.

---- On Wed, 29 Mar 2017 07:37:37 -0400 Balázs Székely via Lazarus &lt;&gt; wrote ----

Online Package Manager shows the packages with "new" icon if the latest

release time was in the given number of days, but shows them with normal

icon if they are older than that days regardless if they are up-to-date

or not.

I my opinion:

- It would be better to show up-to-date packages with normal icon

and

- Show all packages with "new" icon if their last release is newer than

the installed one...

The idea was to notify the user if something has changed in the main repository. With your method non-installed packages wouldn't show up as "new" packages. For example if a completely new package is added to the main repository, you cannot see it as new, which is not optimal in my opinion.Beside if a new update is available a bold NEW text appears in the update column(see attachment).

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus-ide.org/pipermail/lazarus/attachments/20170331/0dbd0143/attachment.html>

GetMem,
it would be very good, if all Json files, for components, will be hosted on one Github repo. Now they are distributed (on NNN servers, etc).
You can make repo for them.

Alexey

Thank you for the suggestions. I agree, the whole system is somewhat confusing, especially if you take into account that the packages can also be updated from the maintainers webpage. I did my best to explain how it works: http://wiki.freepascal.org/Online_Package_Manager#Difference_between_download.2Finstall.2Fupdate

We did try to move all the json to one place, however some of the developers prefer GitHub, others BitBucket or SourceForge. It's very hard to convince somebody to move just for the sake of OPM.

I don't see any advantage. Someone (GetMem?, you?) would have to get the developers access to that repository. The developers needs another password for a system which he does not need for anything else. Now the developer stores the update-json at a place where he already has access to. That's just perfect.

Nope; it works other way. GetMem has repo access. I have my Github account. i clone his repo, make pull request. he sees my PR and applies it.

Werner Pamler wrote

I don't see any advantage. Someone (GetMem?, you?) would have to get the developers access to that repository. The developers needs another password for a system which he does not need for anything else.

It must not be the problem. Many apps use GH, and they need patches on GH, and users make them on GH.

however some of the developers prefer GitHub, others BitBucket or SourceForge. It's very hard to convince somebody to move just for the sake of OPM.

@Alexey
Ok. Let's assume for a moment we move everything to GitHub. First I write to every single developer and ask them to make a GitHub account. By some miracle, everyone agrees and suddenly we are on GitHub. What would be the benefits? Until now I did not have to deal with the external json files, it was the package maintainer job to maintain those jsons. If we implement your idea I have one more GitHub account to take care of, which is not a big deal if there is a clear benefit like:
- OPM becomes faster
- it's more easier to maintain
etc..

Balázs Székely wrote

- OPM becomes faster - it's more easier to maintain etc..

IMO benefit will be. it is "easier to maintain". Each developer which wants to send json,
makes GH acc, then clones OPM repo, then sends pull req. No emails. Zip can be send in GH post.

31.3.2017 22.38 "Alexey via Lazarus"
kirjoitti:

IMO benefit will be. it is "easier to maintain". Each developer which wants to send json,
makes GH acc, then clones OPM repo, then sends pull req. No emails. Zip can be send in GH post.

I don't think so. There is lot's of unjustified hype around Git and GitHub. People claimed the whole Lazarus development process would improve and we would get more contributions by using them. No, actually the GitHub process is much more complex than our current nice bug tracker + patches etc.

I would like the OPM stay agnostic for the source hosting services.

Juha
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus-ide.org/pipermail/lazarus/attachments/20170401/176fc2d9/attachment.html>

...

Please test! Suggestions are welcome.

With online package managers there is the problem of centrally locating all packages at a single URL or repository, correct? So which url will be used? Sourceforge, freepascal.org...
And then you could have mirrors if the main url is down.

Whatever happened to the freepascal package download system at the command line? was that ever implemented? sorry I am not up to date on it..
One advantage of a central package system is that it encourages people to upload their packages, such as like torry.net. Whereas if each guy has his own website and then his website goes down or he quits programming, that package on his site gets lost and people must use archive.org or brute force techniques to find it online...

Github is in a way a central package system but not really as it is not a download tool for binary/releases as much as it's more of a source only package system. Torry.net is really interesting and is the main reason I never used Visual C++ studio because visual C++ never had anything like torry, other than codeproject. Torry never had a way to connect delphi directly to it, like a rubygems or the once discussed fpc package manager tool at the command line.. whatever happened to that idea?

Cool!

Is github okay with this idea though as they are really not a download system like sourceforge or google code...

They are more for source revision and not so much for final release downloads, although, this needs to be clarified as you can indeed just make a tag to release.. But as soon as you start distributing binaries on github, it sort of violates their policy. But, I suppose pacakages are not binaries like a final exe release of a program to be shipped. And lazarus/fpc is fairly uncommon and not very popular so github is likely not going to notice a bandwidth increase on their servers anyway. I think github is against becoming a download website to reduce their costs and focus on being a source revision system, not a distribution system like sourceforge.

2017-04-12 14:56 GMT+02:00 Lars via Lazarus :

GitHub is absolutely fine with uploading binary packages as a release. You can make a release on GitHub where a specific version of your source is accompanied by your binaries.
https://help.github.com/articles/creating-releases/

Regards,
Michalis

@Lars

With online package managers there is the problem of centrally locating all packages at a single URL or repository, correct?

Yes.

So which url will be used? Sourceforge, freepascal.org...

Our own: http://packages.lazarus-ide.org/

Whatever happened to the freepascal package download system at the command line? was that ever implemented? sorry I am not up to date on it..

It was abandoned for a long time. AFAIK is under development again, @joost is working on it.

Yes I agree! This is one of the reason we decide to go with a central repository. Each package can be also updated from the developers webpage(it can be anything: sourceforge, github, personal page, etc...)

2017-04-12 16:26 GMT+02:00 :

Your quote suggests that someone committed large files to his/her GIT repository. This is not a good practice -- aside from GitHub disk space, it makes cloning the repository troublesome (you download all the binaries then, across all the history, by default).

The solution is to follow both GitHub guidelines and general version control systems (GIT or not GIT) guidelines, and do not commit your large binary files to the repository.

This is not how to you make a release on GitHub, see https://help.github.com/articles/creating-releases/ -- you upload binary files using a separate form in that case. This is also the advised way by GitHub to distribute large files:
https://help.github.com/articles/distributing-large-binaries/

Note that I'm in no way an unconditional fan of GitHub (or GIT, for that matter:). I'm only reading GitHub docs and concluding that they are fine with binary releases, when done using their "upload a release" system.

P.S. Cc back to the Lazarus list:)

Regards,
Michalis