Hi

I would like to suggest again a new release policy. It would have timely releases, meaning a new release after a fixed time period, at a predefined date. Some projects have a 3 months cycle but a 6 months would be enough for Lazarus because of more limited resources.

Every release would be basically cut out from trunk. It would be verified to compile and to work with some big test projects. Nothing else. Then it could be called "release x rc1". After one week testing and maybe applying fixes from trunk, it would become the "release x" (without rc1). After that there would be maintenance releases in a timely fashion (or when needed).

The important point is that emphasis would move from the release itself to its maintenance. The challenge is to find all needed fixes for the maintenance and to apply them.
Basically all commits in trunk could be classified as "feature" or "bug-fix". Bug-fixes should be applied also to maintenance branches. The division is not always clear.
Who classifies them, the author of the commit or the branch maintainer? To support this process, any new big feature should stay in its own branch first and be merged to trunk some time after a release (not just before a release).
I could help somehow in this process. I believe that lack of resources is a problem when implementing such a model.

The current situation is frustrating to everybody. Lazarus is evolving and most users must use the development version because it is better than the release version. Some people explain bugs in release version which are fixed over half a year ago.
It is frustrating also to Lazarus developers because the release can happen only when "all bugs are fixed". That is not realistic as we all know.

Juha

2010/10/22 Juha Manninen :

What stops you doing this?

Vincent

I took the liberty to reorder your statements

I guess: yes, you hit the nail on the head.

To support this process, any new big feature should stay in its own branch first and be merged to trunk some time after a release (not just before a release).

I am not sure this is a goof idea.

1) Who defines what is a major feature?
2) we do not want 20 or 30 different branches?
3) New features would take longer, since they were no longer tested. Of course, the end-user a forced tester would anyway no longer be there (since they could stick to releases), but at least other developers would still use all new features.
Personally, I don't mind the occasionally issue with the IDE caused by somone elses new work. I do the same sometimes, and I am always glad to get feedback as soon as possible, as it makes it much easier to fix the issue.

I believe, trunk should stay open for all work (extremely experimental work can and should go into branches, as has in the past).

Then you would have one or 2 maintained branches:
a) the fixes branch as currently
b) the next release branch, which is based on fixes, but also gets features merged as soon as they are deemed stable. Some end-users may even want to stick with this.

And those 2 branches, are what will take a lot of work. If someone is willing to do that work, then I guess that's all it takes.

See above.

There are also some existing test cases. They could even be used on the "b" branch from above, on every merge..., but again: maintenance resources.

Well I think, the commiter can flag them with some sort of hint, the maintainer decides.

Depending on how much work the maintainer wants to spent, I would suggest, that each fix, should stay one or two weeks in trunk, before being merged anywhere else. That way, if it breaks something else we may spot it before merging it.
Or it first merges into "b"-next-release, and if no complaints are received, then into fixes

....

my 2.5 cents
Martin

Vincent Snijders wrote

What stops you doing this?

It would be a competing release system if I start to do it by myself. Besides it is a big job and needs support from the whole development chain. Namely I am thinking the use of branches for new features, and deciding which trunk commits should be applied to maintenance releases.

Maybe I should ask from other projects how they coordinate things. Or maybe somebody here knows.

Also, there should be a recommended release for users and for Linux distro maintainers. The very latest release would not be the recommended one in that model.

Vincent

Juha

Trunk should be used for this, otherwise it will not be tested thoroughly. I don't want to start searching for branches that contain new things, and what if 2 features I want to test are in different branches ?

I am not a chimpansee, jumping from branch to branch, after all... :)

Michael.

No, my idea was that only a new feature which is guaranteed to be broken at the time of next release date, is kept in a branch until the release. Then it can be merged and tested by all trunk users. It will make it into the next release. For example if you start a new "Enhanced Smart Tabs" feature a month before a release, and it will surely mess everybody's code at that moment, then it should go to a branch.

There would not be very many such branches. It would just be a compromise between the current situation when everybody must use the development version and suffer from its broken stuff, and the goal to make a "perfect" release when "all bugs are fixed".

[...]

I had a different idea here (if I understood your text right).

The next release would be based on trunk only. It means it would not be stable and well tested. Later it would have maintenance-patches and -releases and would become more stable.
Having a certainly broken feature in a branch and excluding it from release tries to make the initial release more stable than the trunk currently is. The release maintenance branches would get only bug fixes, no new features.

This is all about finding a good compromise. The current situation is not good.

Well I think, the commiter can flag them with some sort of hint, the maintainer decides.

That would be good.

Depending on how much work the maintainer wants to spent, I would suggest, that each fix, should stay one or two weeks in trunk, before being merged anywhere else. That way, if it breaks something else we may spot it before merging it.

Makes sense.

Juha

Michael Van Canneyt wrote

Trunk should be used for this, otherwise it will not be tested thoroughly. I don't want to start searching for branches that contain new things, and what if 2 features I want to test are in different branches ?

See my reply to Martin.
A branch would be only for certainly broken features at the time of release, making the initial release little more stable than the trunk is now. The branches could also be left out from the plan if they are a problem. The main idea was a timely release + its maintenance.

I think most people agree that the current situation is not good. People are waiting for a "perfect" release while in practice using the unstable development version?

I am not a chimpansee, jumping from branch to branch, after all... :)

:-)

Juha

Juha Manninen wrote

emphasis would move from the release itself to its maintenance.

Exactly.

This summarizes the whole problem of the Lazarus release policy and the only proper fix for it in one sentence.

Releases should be maintained *after* the release and not before it or there will never be any know you are all perfectionists, I am one too, perfectionists strive for perfection but at the same time know exactly that it won't ever be reached. Release is always subconsciously associated with perfection, so it will never be allowed to happen. This cycle must be broken.

Also I suggest replacing the 0.9 in the version number with 1.0 *immediately* and *without* further thinking about it. This would IMHO be the most important and overdue patch of all and it would not cause any regressions! And then make a release next week! It would remove this psychological barrier (and all the discussions that are related to it) and allow the project to continue again.

Bernd

Op 2010-10-22 20:19, Michael Van Canneyt het geskryf:

I don't want to start searching for branches that contain new things, and what if 2 features I want to test are in different branches ?

$ git checkout -b work
$ git merge feature1 feature2

:-)

'work' is the branch you want with whatever features you need. 'feature1' and 'feature2' are the branches with new (large'ish) features. I've been working like this using the git mirror for almost a year now. No problems, and it's really quick to update or reset (sync to trunk) the work branch.

Regards,
- Graeme -

Too much work, because it requires me to know in which branches the features are.

svn update

is much shorter.

Michael.

True, but it still requires me to actually know the names. That means looking them up.
So

svn update

is still much shorter and easier.

See also

http://www.marxists.org/archive/lafargue/1883/lazy/index.htm

;-)

Michael.

Op 2010-10-25 09:25, Michael Van Canneyt het geskryf:

Too much work, because it requires me to know in which branches the features are.

That only occurs if they don't name the branch appropriately. For example, a branch named 'michaels_work' is useless to anybody else. Compare that to a branch named 'ide_build_modes' or 'fpdoc_rtf_hyperlink_support'.

svn update is much shorter.

I'm lazy too, that is why all my usual git commands for Lazarus repository are in a script. So I simply run: updatework.sh :-)

Regards,
- Graeme -

Real lazy people like me do an
svn up
only or have even an u script :)

Hello Juha,

[...]

good idea, but what actually stops us from adopting the Linux model ?

Each developer has his own git tree and lets everybody know to pull from it (and what to pull) when time comes.

Then, the 'boss' merges 'new features and regfression- and bugfixes', into trunk, also cutting-edge development happens there.

Then, everybody interested in fixing regressions also have a git tree where they commit, and it is (if applicable) also applied to trunk. As this tree is 'always stable' i.e. the API and ABI are frozen there, this is what most non-developer people would use and report regressions to.

I know the major difference is that Linux (the kernel) has (relative to FPC) unlimited developer resources, but this model is also infinitely more scalable. You can pull 'cherry-pick' from trees, not hop from tree to tree like an ape ;)

That model would happily accommodate every idea somebody wants to push, and actually let people have them, test them and so on.

Also it makes things like priority/security patches hit mainline very fast.

Also, the first step is already there : we have a git mirror. And there is tortoise-git for Windows, as well as several types of gui for Linux.

And if you don't want to use git, what stops you from organizing it around several svn repositories and emailing patches to each other?

Juha

Lukasz

Such schemes only work in retrospect. With 20 people all having a couple of pet projects you are quickly talking hundreds of branches.

This might be doable for the Linux kernel, but they have a whole infrastructure of people that do nothing but code review and merging patches. (and quite some of them are paid to do it)

It doesn't really fit to our workflow.

Aside from that, this would work fine with SVN too, it is the workflow that is the question here. Not the VCS.

Lukasz Sokol wrote

[...] good idea, but what actually stops us from adopting the Linux model ?

About 6 dedicated paid people (and a whole bunch more volunteers) to manage all the branches and review and administrate each patch to reitegrate all the various patches into a few production trees.

The Linux kernel workflow, being security and stability sensitive in nearly every line of code is totally different from a normal project, and extremely costly in manpower. Moreover it relies on some central (bosses) being able to put in a lot of time into the project on a regular, guaranteed basis, which is hard if they are not at least partially fulltime.

Op 2010-10-25 10:22, Marco van de Voort het geskryf:

Such schemes only work in retrospect. With 20 people all having a couple of pet projects you are quickly talking hundreds of branches.

So what's the problem with that? Is there a limit of branches that SubVersion can handle? If so, use a better tool for the job.

It doesn't really fit to our workflow.

Well the workflow is under question here, and that it what needs fixing. I raised this isuse before, now Juha is raising this issue. Good luck to him changing the Lazarus team's mind. ;-)

Regards,
- Graeme -

2010/10/25 Graeme Geldenhuys :

No, there is a limit on the number of branches you - sorry, I mean *I*
- can overview. For me, two long living branches are already too much.

Vincent

Op 2010-10-25 10:27, Marco van de Voort het geskryf:

About 6 dedicated paid people (and a whole bunch more volunteers) to manage all the branches and review and administrate each patch to reitegrate all the various patches into a few production trees.

Maybe the Linux project was not the best example, because as you say, it has commercial funding. So rather look at the Git project then. It has many many branches (though only a select few are made public on a public server
- for end-user convenience). The Git project also has a "stable" branch, an "experimental branch", a "fixes / maintenance" branch etc. All managed by a single person (Junio) - not being paid for the job. And that single person still has time for his own development too.

All I'm saying is, choose the right tool for the job. Not every tool is good in every situation. So maybe SubVersion is partly to blame for the workflow issue Juha mentions. Just a though.

Regards,
- Graeme -

Hi Marco,

I know the FULL Linux workflow won't fit - but what if it was scaled to our needs and manpower ?

One production (stable) tree.
One devel tree.
Peoples' private trees where the maintainer(s) of devel tree can pull from. Communication between developers to let each other know what each branch contains. Regression fixes always applied to top of both trees. Top of devel tree periodically becomes merged to stable tree.

Juha's idea is actually exactly that if you read it carefully.

Lukasz

It's not the tool. Tools are just that - tools to do a job. Redhat still uses CVS, and they're doing just fine with that.

It's the amount of people needed to actually do the job which is the problem. No tool will alleviate that.

Whether Vincent has to type 1 or 2 commands to do a merge is not the issue. That's just a couple of seconds.

It's the code review after this/these command(s) that is really time consuming.

Mihcael.

And? Git is only 300k LOC, it requires only a part time branch manager. Fine. The Linux kernel is 12M LOC: six full time branch managers. This narrows it down how much branch managing the 1.5M LOC of Lazarus would require: almost one full time branch manager. You? Juha? Anyone?

Op 2010-10-25 10:35, Vincent Snijders het geskryf:

No, there is a limit on the number of branches you - sorry, I mean *I* - can overview. For me, two long living branches are already too much.

Distribute the work-load. The developer working on whatever branches are responsible for keep it up to date. Why do you think you must do that? All you (meaning the person maintaining the key public branches like Fixes, Trunk etc) need to do, is merge those "feature" branch into others. If they don't merge, revert the changes, email the maintainer of that branch to get his work in order.

The key is to delegate! :-)

Regards,
- Graeme -

Let me quote you from a few days ago on fpc-other:

"I've taken the following stance with the Lazarus team. I contribute, and forget about the patch. They can do with it as they please, I did my part."

The problem is keeping track of them. See my other reply in this thread.

Well, I only see "do as Linux" combined with "GIT solves everything", and "more branches is better".

What I haven't seen:
- the problem
- the solution
- practicality of implementing the solution

which is the problem with all the GIT noise. Much throwing in gratituous principles, little practical implementation.

The few longliving change branches are already in SVN (cpnewstr, llvm and Dodi's branches etc) and the respective authors do this, and are responsible for merging them into trunk, or delivering a clean patch set so that others can do.

Op 2010-10-25 10:43, Michael Van Canneyt het geskryf:

It's the code review after this/these command(s) that is really time consuming.

I guess this is where the tiOPF project got it right with it's extensive test suite. Each new feature or fix is accompanied with one or more unit tests to prove that the patch does what it said it will do. A unit test to show the issue (the test should fail) is applied first. Then after the feature/patch is applied, that test should pass (including all the other previous tests too - obviously).

Unit testing a GUI application is considerably harder (but not impossible), but testing a framework like LCL is not that difficult.

Regards,
- Graeme -

Just ask the people requesting such a micro- and feature branch based workflow. It seems they think it is worth.

Michael Van Canneyt wrote

Redhat still uses CVS, and they're doing just fine with that.

No, they do not.

http://fedoraproject.org/wiki/Using_git_FAQ_for_package_maintainers

You may be willing to use outdated tools yourself, but do not accuse others of doing that ;-)

Op 2010-10-25 10:46, Florian Klaempfl het geskryf:

narrows it down how much branch managing the 1.5M LOC of Lazarus would require: almost one full time branch manager. You? Juha? Anyone?

So where do we discuss remuneration?

Regards,
- Graeme -

We already have that. (except when migrating from one stable to the next, then there are a few more on paper, though usually the "old" stable branch is then quite inert)

Peoples' private trees where the maintainer(s) of devel tree can pull from.

I don't see that need. That puts only strain on the maintainer to keep track of private trees.

Currently there are private branches to do major development. I don't see a reason to scale this up for all kinds of minor development, if this can be communicated using a patch in mantis.

Communication between developers to let each other know what each branch contains.

There is no reason for this, until we really start multiple levels of stable branches. But even then, hardly any private branches come into the picture, just people merging back features from the more liberal stable branch to the more conservative stable branch.

The whole idea of private branches is that core does not need to know about it, IOW delegation. When it is finished, people supply cleaned up patches in mantis, and it doesn't matter if these patches come from a private repo, or that sb ran DIFF of an old checkout with a new one.

Regression fixes always applied to top of both trees. Top of devel tree periodically becomes merged to stable tree.

This is all that already happens today (and for about a decade). Since mergetracking was initiated, it is a bit easier to keep track of patches to merge (http://www.stack.nl/~marcov/fpctomerge.txt is a list generated by SVN of trunk patches not in fixes)

During release times, there is another branch/tag, of the upcoming release that is likewise connected to fixes. (so merge order is trunk->fixes->release)

Juha's idea is actually exactly that if you read it carefully.

The whole point of the others is that the current setup already strains resources, specially during release preparation times.

Maintaining any extra branches (e.g. splitting trunk into "more stable" and "less stable") requires a management effort, which is why it is not done. (and it could be perfectly done with SVN. It is the work and the management that is the problem, not the tool)

Maybe I am misunderstanding the discussion, but that is because all these proposals are very vague, rely on workflow of external projects, and the people don't seem to have a clue about the current FPC/Lazarus workflow, and where the bottlenecks are.

Op 2010-10-25 10:50, Florian Klaempfl het geskryf:

Let me quote you from a few days ago on fpc-other:

And did you bother to ask why I do that? Past history explains that, and the currently flawed workflow of the Lazarus project.

Regards,
- Graeme -

Marco van de Voort wrote

What I haven't seen: - the problem

Lazarus project is unable to do releases in a timely manner.

- the solution

Make a releases time-based, as suggested at the start of the thread.

- practicality of implementing the solution

Most other open-source projects are doing that.

which is the problem with all the GIT noise. Much throwing in gratituous principles, little practical implementation.

While it is true that git may help, time-based releases can be practiced using subversion too. All that is needed is to avoid Debian trap of striving for unattainable goal of the "perfect" release.

I guess you have some cheap excuse like that they didn't accept some patch from your granny.

Past history explains that, and

Indeed ...

the currently flawed workflow of the Lazarus project.

So how much do you contribute for the remuneration of the branch manager?

Op 2010-10-25 10:52, Marco van de Voort het geskryf:

The problem is keeping track of them. See my other reply in this thread.

Again, use the right tools for the job. Git (I really didn't what to say this again) makes this easy.

What I haven't seen: - the problem

Good lord, read you damn emails. Read my messages in the past with a similar topic. Read Juha's original post. Listen to what the many Lazarus users post quite often on this mailing list.

- the solution

Again, read some responses in this and many other similar threads.

- practicality of implementing the solution

like with everything else, things evolve and change over time. KDE used CVS, then SubVersion, then Git. Linux project, a similar thing. tiOPF project used emailing patches, CVS, then Team Coherence then SubVersion. Thousands of other open source project are changing there ways over time - as newer and better tools become available. FPC and Lazarus projects still seem oblivious to this fact. God only knows what a struggle it must have been to come to the consensus to move from CVS to SubVersion - thank God it was before my time.

Regards,
- Graeme -

Op 2010-10-25 11:12, Florian Klaempfl het geskryf:

So how much do you contribute for the remuneration of the branch manager?

You understood me wrong, I was putting my name up for the position of the branch manager (fat chance that will ever happen), not offering to pay somebody else. So the remuneration question was, what will you pay me to do the job. After all, those 6 dedicated branch managers you mentioned from in the Linux project get paid.

Regards,
- Graeme -

Op 2010-10-25 11:10, Alexander Klenin het geskryf:

+1
Finally somebody that understands the problem correctly. Even staying with SubVersion is fine, but improve the management of the project. That would be a damn good first step.

Regards,
- Graeme -

Did you miss again my irony tags?

I was putting my name up for the position of the branch manager (fat chance that will ever happen), not offering to pay somebody else.

You complained the workflow is flawed, so you should do anything against it. Complaining is easy but it's waste of time to do it again and again if nobody comes up with a practicable solution.

I guess you prepared also a list of people who volounteer to do the boring job of release building?

2010/10/25 Florian Klaempfl :

I volunteer for actual building.

Vincent

I just said 'it takes a lot of work', and you respond by making it even more work ? :-)

Unit testing a GUI application is considerably harder (but not impossible), but testing a framework like LCL is not that difficult.

To me, these statements sound contradictory, since LCL is a GUI framework ?

How can one test that setting

   MyLabel.Caption:='Some UTF8 string with weird characters';

will actually display the correct string ?

But it is beside the point. I do not believe that a testsuite will actually make it less work to manage the patches.

Michael.

hello,

All of this has already been discussed to death. Timed releases have already been proposed (and even accepted) about 2 years ago, but then there was always some kind of problem which actually prevented it from becoming a reality.

It is obvious that it is a good time for a new release now. It is also obvious that every 6 months is a good amount of releases and this time we couldn't make it because fpc 2.4.2 became a requisite.

As Vincent already pointed the real issue is that we need someone to do extensive testing and verifying that a revision X (or a branch Y) is ready to be released. Testing should involve all platforms and check the basic funcionality.

bye,

Op 2010-10-25 11:26, Florian Klaempfl het geskryf:

> You understood me wrong, Did you miss again my irony tags?

Clearly Thunderbird's junk mail filter is too good. ;-)

Regards,
- Graeme -

Op 2010-10-25 11:37, Felipe Monteiro de Carvalho het geskryf:

As Vincent already pointed the real issue is that we need someone to do extensive testing and verifying that a revision X (or a branch Y) is ready to be released. Testing should involve all platforms and check the basic funcionality.

But the point is that with any amount of testing a specific revision will NEVER be 100%. That is what maintenance releases are for - to fix this little things that was overlooked. Release quick, release often. The important bit being that maintenance releases must also happen on a timely fashion while bugs are detected, or stop when a new full release replaces it. Maintenance releases, like full releases should not be limited to once a year.

Regards,
- Graeme -

Op 2010-10-25 11:37, Michael Van Canneyt het geskryf:

I just said 'it takes a lot of work', and you respond by making it even more work ? :-)

Again, the person applying the patch simply need to do just that. Glance over the patch to spot if the author of the patch was relatively sane, apply the patch, run the test suite. If fine, commit.

The 'commiter' doesn't write the test cases, that's the job of the 'patch author'. The patch author is the one doing most of the leg work: making sure he/she has a test case to prove the patch, make sure it can be applied without conflicts etc.

To me, these statements sound contradictory, since LCL is a GUI framework ?

of the LCL behaviour can indeed be tested for. Order of firing events, making sure events actual do fire, the various UTF8 and file wrapper functions etc, etc. There is even a Delphi + DUnit project on SourceForge which is dedicated to testing of GUI applications (events firing, tab order, button order etc.)

Regards,
- Graeme -

Actually it isn't impossible to do this. What I do all day is exactly writing automated test suites and semi-automated ones.

The most common solution is doing a semi-automated test. The test suite does operation X and then asks the tester if the result is the expected. He chooses Yes or No. You can also make automated versions which use for example image-to-text technology.

Usually both tests are necessary, both automated and semi-automated, and also a few fully manual tests are required.

But it is beside the point. I do not believe that a testsuite will actually make it less work to manage the patches.

That is correct. Only for very intrusive changes a full test suite should be run for each patch. Testing is done more statistically, running the test periodically, let's say once a day. Not once per commit. Even in companies with lot's of full time Quality Assurance Engineers.

Anyway, having a test suite is good, but not indispensable.

I will try to improve this when I find time, but it won't solve the issue that we need more people working on the task of verifying that a release X or branch Y is ready for releasing.

Graeme Geldenhuys wrote

But the point is that with any amount of testing a specific revision will NEVER be 100%. That is what maintenance releases are for - to fix this little things that was overlooked. Release quick, release often.

Releasing without enough testing also causes issues on it's own. 0.9.28 was released with a bug introduced a few days before the release, it was a crash at running a new project or something similar. Then 0.9.28.2 has to be release very shortly after that. Not very good, and clearly shows that we need more people responsible for doing testing of the basic functionality at least before releases.

Marco van de Voort wrote

The whole point of the others is that the current setup already strains resources, specially during release preparation times.

I.e you needed a volunteer to step up and propose himself to do this, and somebody did. (i.e. Juha, Vincent)

They are vague, because they need to fit all interested peoples' work flows.

They don't rely on work flow of other projects, they take an example from them
- they try to get in whatever-good people invented somewhere else, to benefit our cause.

Is the current workflow described somewhere detailed enough that comments can be made?

Lukasz

Juha Manninen wrote

I could help somehow in this process. I believe that lack of resources is a problem when implementing such a model.

So I propose that you become our official pre-release tester.

I can improve the automated test suite and create semi-automated tests to support testing.

Vincent can do the actual building.

You would be responsible for saying that revision X or branch Y is ready for testing and for running the full test suite in all platforms for this revision / branch.

What do you think? If accepted, then the discussion should move out of here into the internal mailling list.

bye,

Op 2010-10-25 12:13, Felipe Monteiro de Carvalho het geskryf:

Then 0.9.28.2 has to be release very shortly after that. Not very good, and clearly shows that we need more people responsible for doing testing of the basic functionality at least before releases.

True, but realistically, Lazarus users tend to use released version more often that "development" versions like Trunk. Sad I know, but currently it seems like fact. So with saying that, it seems that released version get more attention and testing, so the importance of early and often maintenance releases (and even full releases) are greater. This should help address the "things are not being tested" in Trunk issue. Plus, having releases more often makes the project look like it is very active and fast paced - this is good PR.

Regards,
- Graeme -

You are talking about stuff that merely creates an overview of patch(sets)'m talking about the actually meaning of a patch, what problems it can cause, its regression testing and the like.

Yes. And as said they are all likewise vague, hide behind comparisons, incidents etc.

That is exactly the point. The main proponents of SVN (Florian and Peter), at the time prepared the migration, provided a lot of info, examples of usage, suggested workflow etc. The same when Jonas investigated merge tracking.

They didn't just point to some other project, but applied knowledge to the actual buildup of the project.

Alexander Klenin wrote

> What I haven't seen: > - the problem Lazarus project is unable to do releases in a timely manner.

Major or minor?

> - the solution Make a releases time-based, as suggested at the start of the thread.

This has been discussed many times, and as current release manager of FPC I've tried to do this. (6 months).

The problems are always that people only start moving when release building is already upon them. And then these people have no time, and then those.

Then other people are discussing GIT on the maillist instead of testing 2.4.2-rc1.

> - practicality of implementing the solution Most other open-source projects are doing that.

That is not an argument, only an indication.

I hate the debian situation too, but there is a reason for this: both FPC and Debian are entirely volunteer driven efforts.

Neither.

This indicates a problem with the release process. There should be no need for more than one person to do the release. Much less in FPC, where, unlike Lazarus,
a more-or-less comprehensive regression testing should be possible. (Yes, I am discussing "ideal world" here, but every modern language/compiler does this, so clearly it is possible).

Then other people are discussing GIT on the maillist instead of testing 2.4.2-rc1.

True, I will now go and test it some more ;-)

This may be the cause, but not the reason.

Another, maybe more relevant case study is a Perl project. A couple of years ago it was in the position very similar to the current FPC one:
outdated, unextensible code base,
fear of changes due to backwards compatibility burden, glacial release process,
rapidly losing ground to competing languages,
even endless process-oriented flamewars
instead of technical discussions in the mailing list ;-)

Perl porters managed to overcome most of the problems (except, of course, language competition)
and seriously invigorate Perl development by
1) Streamlining release process, so a single person can do a release in a single day.
2) Mandating time-based releases
3) Moving to a better SCM

It should be noted that FPC problems are worsened
by the lack of tests, while Perl's test suite is one of the best among the language implementations.

No, we would need heaps of volunteers. Since this only covers the release prepations, but also a lot more builds need to be done by the respective platform maintainers.

Is the current workflow described somewhere detailed enough that comments can be made?

Currently there is a trunk where major development goes into. Committers doing largescale development that could break trunk for long stretches do this in private branches. These are then merged back to trunk (using SVN, or manually, or by different means, e.g. by doing it in phases.)

Non-committers communicate via patches in the bugtracker, or more informal means.

Before a major release, the trunk is branched to a fixes branch from which releases are made (currently 2.4), and each release is branched from this fixes branch.

Bugfixes and features are then merged back from trunk to the fixes branch.

In general most library features are merged, except the ones related to new compiler features. Patches to compiler/ (whether fixes, features new targets etc) are much more selectively merged. This is because these are more likely to upset the "stable" aspect of the reulease branch.

Before a release there is always at least one release candidate. This is mostly to flex the packaging muscles. In this stage usually many problems are detected (like a new windows version getting popular and causing UAC problems), and in general in this stage effectively release oriented development is done and merged back.

Usually the month after a release sees some release-engineering commits (often more definitive solutions to quick fixes), and then the line goes flat. release engineering fixes are almost never externally supplied, and are always done by core members and platform maintainers (then I define core a bit loosely as everybody having commit access on the tree, including many lazarus devels)

In the past doc building has been a bottleneck, but due to recent interest in fpdoc, improvements in fcl-xml, and less trouble with the latex2html conversions, this is not as bad as it used to be.

Often platform developers really only start test-building releases when the final deadline for a RC is due. One can continue without the other then of course, but one could wonder what good a release with only OS/2 and FreeBSD (real life 2.4.2 example) would do.

This also demonstrates what is needed hardest to accelerate releases: 3rd party involvement in release engineering. All resources are open to my knowledge, and there is quite some info in the wiki, and more than enough people willing to answer questions.

Interested users could start testbuilding releases, and keep release engineering alive outside the typicaly release building intervals.

Specifically:
- building rpms (and testing them on both fedora and Suse (Mandriva?)). Maybe even splitting rpm releasing from one general RPM to one per distro?
- same for .deb (debian ubuntu)
- windows installer (each release a few issues are wrinkled out, but it is still far from perfect)
- doc building
- the integration of GDB into libgdb sometimes is a problem too, specially for the linux distros.

So the question can be short: do any of the people in this thread actually tried to build a FPC or Lazarus release?

Op 2010-10-25 12:58, Marco van de Voort het geskryf:

Then other people are discussing GIT on the maillist instead of testing 2.4.2-rc1.

It's not clever to generalize, because you have no clue what others are doing. I made that same mistake in the past.

As a matter of fact, I have switch to using FPC 2.4.2-rc1 on my work development system (linux 64-bit) since the day you announced it. I switched our Windows development to FPC 2.4.2-rc1 + Lazarus (the Vincent release) the day he announced it. So at least here at Master Maths we are using 2.4.2-rc1 every day.

Regards,
- Graeme -

No doubt. But I doubt random shuffling

There should be no need for more than one person to do the release.

So you have machines and installs for all platforms, contacts with fedora,debian,fink, follow the Mac lists about new news etc?

Sure, centralizing it in one person makes for shortest communication lines, but it is not a practical solution.

Much less in FPC, where, unlike Lazarus, a more-or-less comprehensive regression testing should be possible. (Yes, I am discussing "ideal world" here, but every modern language/compiler does this, so clearly it is possible).

The testsuite is run nightly for a decade or so?

> Then other people are discussing GIT on the maillist instead of testing > 2.4.2-rc1. True, I will now go and test it some more ;-)

Not purely a joke. Feedback has been on an alltime low.

> I hate the debian situation too, but there is a reason for this: both FPC > and Debian are entirely volunteer driven efforts. This may be the cause, but not the reason.

I think so. The other main distros are hosted at companies with huge buildclusters that have dedicated admins.

On the otherhand, when I do a major release, I have to dust of some machines with older freebsds, sometime fix the hardware etc.

Do they actually _do_ releases, or do they only release an source archive?

Keep in mind that unlike many other projects, we need to do the _full_ release engineering. Distro maintainers are unlikely to dive into Pascal.

cron job (every night, including rebuild and restart)?

wake up in the morning and the latest IDE is already running ;)

... or broken :-)

Not such a good scheme. My monitor has the lazarus IDE burnt in.

This is not a joke; you can clearly discern the component palette, toolbar etc :-)

Michael.

That can be said about every software project, look at http://thread.gmane.org/gmane.linux.kernel/1028235 for what the 'masta haxx0rz' think of the 'small people' that like to use old stable foo.

There is that (exaggerated) sense of instability using a 'development' branch - you never know which part of the quilt are 'they' going to pull from under yourself, and need to be aware of every corner case. It is exhausting and needs many resources, e.g. time. And in constant-devel branch this will always be the case;

Lukasz

Regards, - Graeme -

Zitat von Michael Van Canneyt :

Maybe someone should create a screensaver package to randomly rearrange/colorize the IDE over time. ;)

Mattias

That kind of reasoning leads to longer times between releases and progressively becomes worse: You don't want to release because you want more tests before but, as releasing become increasingly a greater effort, you do it less often. Worse, the bigger delays between releases worsen the creeping bugs that will surface when the release is exposed to a different crowd.

If the actual release isn't a lot of effort, a botched release can be solved by a new release and the bad one is quickly forgotten (if the new one came up fast enough).

So a bad 0.9.28 is not a problem if a good 0.9.28.2 can appear a few day latter.

Paulo Costa

I had this idea and I just want to throw it out here. What about taking the Synaptic approach to Lazarus with Packages and LCL components. Have version dependencies and present and Automatic updates and Manual updates source get from SVN and re-build as needed?

On top of that - borrowing the Ubuntu approach where you schedule 2 major releases a year and all others are just minor version updates along the way.

Alexander Klenin wrote

to the current FPC one:

[...]

Ok, then we need first some sponsors for hardware :) We're releasing binary releases and not only a blob of source code. Then we need somebody with deep knowledge about windows, all unixes, OS/2, DOS etc. Further, he needs to know all unix package systems and is able to fix errors in all the different installer information files from inno setup to the home made DOS installer.

It should be noted that FPC problems are worsened by the lack of tests,

Really? What do you miss in the compiler testsuite?

Op 2010-10-25 13:26, Marco van de Voort het geskryf:

Since this is the Lazarus mailing list (not FPC), I'll make the huge assumption that Alexander was referring to the Lazarus test suite, and not the FPC test suite. If we take that assumption as correct, then the Lazarus project is in a worse state than I first expected.

- No nighlty or hourly tests are run.
- When I ran them the other day, 170 out of 250 tests failed
- 250 tests for a project as large as Lazarus IDE + LCL is minute. In comparison, tiOPF (which is a much smaller project) has around 1700+ tests and still growing.

As for regression bugs in Lazarus... oh, don't get me started on that issue. The Lazarus project has the worst track record I have ever seen.

Regards,
- Graeme -

In my opinion not entirely true; but it ends as a catch 22...

1) a) to many botched release will put some people off => bad b) a fixes branch needs a release manger, who maintains it. Without a release manager, there will not be a fix-release...
2) To few releases will put other people off.

Anyway the problem is not that we want certain bugs/regression closed before release.
** The problem is the missing release manager. **
Because no release branch is forked ( or if it was, it would not be managed), by the time the regressions are fixed other regressions are there.

If we have a release manager, who cares about a release branch, then a reasonable amount of quality can be archived in time (or within little overtime).

Only if we have such a release manager, then we can judge, if the release standards are to high.
Before we have a release manager, that is all speculation, that will not lead anywhere.
(Yes there are other opinions, already expressed, and even given examples => please do not repeat them. I really did read and consider them)

The end is either:
- Someone takes that position
- there will be another 100 mails on this, and it will end without anything changed

Well, I think you dehumanize the developers that are currently doing the job. It is not like they are braindead droids that can only work this way.

Worse, the bigger delays between releases worsen the creeping bugs that will surface when the release is exposed to a different crowd.

There is some truth in this, but that is a chicken and egg situation. If new versions are not adopted for a while because each time a differnet little thing makes it unusable for a certain group, you end up with multiple groups clinging to old "golden" releases.

If the actual release isn't a lot of effort, a botched release can be solved by a new release and the bad one is quickly forgotten (if the new one came up fast enough).

A release will remain a lot of effort. The building and platform coordination is what takes time, not the VCS work.

So a bad 0.9.28 is not a problem if a good 0.9.28.2 can appear a few day latter.

Some maintainers might be incommunicado for several weeks. Holiday periods are difficult for the same reasons. (which is partially why 2.4.2 is so delayed, july-august nothing happened)

This cycle can certainly get broken, e.g. by multiple maintainers per target, and not only doing release engineering around release periods.

But that needs new blood, and the question is simple: who in this thread has ever built a release? A whole release (so installer, or rpm/deb and tried to get it accepted by Fedora/Debian etc).

THAT's where the bottleneck is. The branches etc work is one afternoon.

I think you overrate importance of synchronized binary release for all platforms. I thin kit is ok to postpone release of binary packages for exotic and outdated architectures.

> It should be noted that FPC problems are worsened > by the lack of tests, Really? What do you miss in the compiler testsuite?

First of all, I miss structure ;-)
Is there any way to check if some feature is covered by a test? For example, I would like to know if there is a test covering object initialization bug discussed yesterday -- how do I proceed?
Secondly, it seems that some tests are under-checking, especially those in tbs and tbf directories.
tbf test check only the fact that compiler fails,
not caring if it fails with the correct error message. Even worse, tbs tests check only that the compilation succeeds, not caring if it produces correct result.

It's not that. With 2.4.2rc1 it was the major platforms that were late, not the fringe ones. Logical, since e.g. OS X with 5 archs has the heaviest burden, and Linux suffers from constantly changing packaging details. (even within one distro, over half years-years time, libraries get renamed or splitted etc)

> Really? What do you miss in the compiler testsuite? First of all, I miss structure ;-) Is there any way to check if some feature is covered by a test?

Is there a list with features and codenames for them? But in general grep goes a long way.

For example, I would like to know if there is a test covering object initialization bug discussed yesterday -- how do I proceed?

A "do what I think" button :-) ? How would you formulate such a request?

A simple tagging system is probably doable, do you volunteer? Not only to implement it, but tag the thousands of existing files?

In short, we don't need people that say what can be done, but people that actually do things. Preferably without rearchitecting everything first.

Alexander Klenin wrote

> Really? What do you miss in the compiler testsuite? First of all, I miss structure ;-) Is there any way to check if some feature is covered by a test?

I'am not aware of any query language which could cover this.

For example, I would like to know if there is a test covering object initialization bug discussed yesterday -- how do I proceed?

With some brain: a quick browsing of the failures on the primary platforms shows that there is apparently no failing test regarding this. Conclusion: it is no covered.

Secondly, it seems that some tests are under-checking, especially those in tbs and tbf directories. tbf test check only the fact that compiler fails, not caring if it fails with the correct error message.

Nice theory but 15 years of production compiler showed that regressions regarding wrong error messages are no problem.

Even worse, tbs tests check only that the compilation succeeds, not caring if it produces correct result.

Maybe you should really run a
grep -i halt
in the tbs dir and think and conclude and stop spreading FUD.

sent from my Telstra NEXTG? handset

sent from my Telstra NEXTG? handset

the first thing i see is "too many branches, too many trees" and there's also the recent discussion about "need to know the names of the branches/trees for the features" and "i have an u script" :P ;)

FWTW and a really huge :)

;)

sounds like there should be some pruning or grafting and the making of a new tree which can grow and branch like real trees? maybe? :)

yes, there's a large smiley there and more to come... believe me, i know the problem as i've run my own company three different times and have always had problems trying to find people that can handle parts of it for me... in the end, i ended up doing most of it myself because i never really found those who could do what needed to be done the way i wanted it done... i don't envy anyone in that position, for sure :) ;) ;) ;)

or even a 0.9.30 if .29 is devel version ;)

was just a warning to a possible and very _human_
reaction that can lead to a lower frequency of releases.

Yes, but at least it their choice not to use a more modern release. Probably, the bulk of the users would jump to the latest version. That would give more "testers" than the current trunk can ever get. (Provided that there are real improvements: more features/less bugs from release to release)

An heretic thought: Why should a release be made to all platforms at once? The release cycle could be different for different platforms...

Paulo Costa

My bad. I stupidly assumed the above from reading readme.txt, without looking at the actual code. Sorry.

So nothing which cahnges the fact that the feature branches in the archer git repository are a huge mess. All the archer-... heads are not part of cvs but only in git.

This is exactly where my git-nightmares are coming from.

Well, as you're such a bug Git fan. And you want a better debugging-environment....

Just checkout the archer/gdb project, you can find it at:

git://sourceware.org/git/archer.git

Now you only have to search for all the pascal-related fixes splattered all over the place in all the feature-branches. Just do:

git merge feature1 feature2 feature3 ...

Well, you have to look up the names of all these feature-branches, offcourse. And you'll notice they won't merge that easily...

But you can create a much better gdb-environment that way.

Oh, and ... good luck!

Joost.

That's only for about two months. And only for Fedora. Redhat itself is still converting... So MvC knowledge was a bit outdated, you don't have to attack him like this on that.

It is nice to see, though, that they have created their own layer around git, to avoid some of it's pitfalls we discuss here now and then too.

Basically, they use Git as if it was cvs. (No staging and caching. commit and pull are combined. No feature-branches etc.)

Joost.

Op 2010-10-27 14:13, Joost van der Sluis het geskryf:

Basically, they use Git as if it was cvs. (No staging and caching. commit and pull are combined. No feature-branches etc.)

In that case they should have stayed with CVS, because they are killing off all the best features of Git.

Regards,
- Graeme -

Joost van der Sluis wrote

But you can create a much better gdb-environment that way. Oh, and ... good luck!

Graeme just ignores only slightly more complex questions about git (typical fanboy behaviour) so don't expect too much.

q.e.d.

Op 2010-10-27 15:02, Florian Klaempfl het geskryf:

Graeme just ignores only slightly more complex questions about git (typical fanboy behaviour) so don't expect too much.

Oh piss off! I have work to do.

Regards,
- Graeme -

Op 2010-10-27 14:00, Joost van der Sluis het geskryf:

That is a half-baked mirror of their primary CVS repository (from what I can see). That repository doesn't use any features of Git, so what did you expect.

In any case, I have no time for tinkering with C code, or learn such a lot of spaghetti procedural code (as I have heard from so many here). I know just enough C to read it, but not code with it. I'll rather spend the little time I have on a Object Pascal based debugger. Which is what the FPC team should have done from the start. Or improve tiOPF's "log to GUI" class to have the features of Raize's CodeSite Live Debug Viewer.

Regards,
- Graeme -

I see. One has apparently to throw away all the old history to have a full baked git. Even more a no go for FPC and probably lazarus as well.

That repository doesn't use any features of Git, so what did you expect.

What features do they not use? At least they use heavily feature branches.

Op 2010-10-27 15:20, Florian Klaempfl het geskryf:

I see. One has apparently to throw away all the old history to have a full baked git. Even more a no go for FPC and probably lazarus as well.

As usual you don't understand me, and give to the first wrong conclusion - what's new. Simply having a git mirror of a another SCM repository doesn't give you all the features of git. Yes you have the full history, but you don't have rename information, cherry-pick information (like when fixes have been applied in multiple branches), different between patch author and committer, sign off info etc, etc... The GDB project doesn't use Git, they use CVS, so none of the Git features will be in the Git MIRROR repository.

Bottom line is still, I'm not going to learn procedural C language and whatever toolchain comes with it, when I only program in Object Pascal on a daily basis.

Regards,
- Graeme -

You missed the difference between gdb and archer. The archer project is a sort of a fork of gdb. (Although the goal is to merge the changes back to gdb)

Archer uses a completely functional git repository. They only have one one or maybe two branches which are tracking the 'original' gdb sources.

Bottom line is still, I'm not going to learn procedural C language and whatever toolchain comes with it, when I only program in Object Pascal on a daily basis.

That's another thing, offcourse. But in principle you can cherry-pick your gdb-version together and do a './configure && make all' and yuo have your gdb....

Point is that with all those branches, it's really hard to track things and to combine all things you need.

Joost.

Joost van der Sluis wrote

You missed the difference between gdb and archer. The archer project is a sort of a fork of gdb. (Although the goal is to merge the changes back to gdb)

My apologies, it did miss that. I used my web browser to browse sourceware.org domain, and clicked through to the GDB repository to browse it online. That's when I saw the reference to a git mirror, but CVS as main repository.

Point is that with all those branches, it's really hard to track things and to combine all things you need.

And coming from my side with limited C knowledge and no knowledge at all about the GDB project or debugger internals, it would be even harder to know what does what. So I'll leave GDB development to the GDB/Archer team or other C experts.

I took a quick peek at the code, randomly clicking a source unit here and there.... Man am I glad I use Object Pascal and OOP. Procedural code (and C on top of that) just looks to alien for me. [screw what Linus might have to say - somebody quoted a post of his about C and procedural coding a few days back]

Graeme Geldenhuys wrote

Linus might have to say - somebody quoted a post of his about C and procedural coding a few days back]

That was on the git mailing list, and really aimed at c++. To be honest, I find well designed non-oo code much easier to read than oo code (probably because I suck at oo :).

Henry

Take a look at the code of ReactOS. Nearly everything is written in plain C, but it's rather easy to read and understand. :D

Regards,
Sven