Hello,

The FPC team has been recently made aware that a number of routines in the Classes unit are apparently based on code originally from Borland/CodeGear. After someone found a tool to automatically compare source code bodies to look for structural similarities, we improved its support for Object Pascal (it's in svn now) and used it to compare all FPC RTL and FCL units with their Kylix 3 and Delphi 7 equivalents (in so far these exist).

This enabled us to find more cases where code may have originally come from or be based on Delphi/Kylix code. All such identified routines of sufficient complexity and/or likeliness have been or are in the process of being rewritten from scratch using a clean room approach: by someone who is not intimately familiar with the current implementation, and only based on documentation written by other people. I am personally supervising this effort.

This whole issue highlights a quite annoying problem though: one the one hand, it is most certainly our responsibility to ensure that all committed code can be legally licensed under FPC's modified LGPL license, and in particular does not originate from Delphi/Kylix sources. On the other hand, we have to avoid reading Delphi/Kylix code as much as possible since by doing so, we may become "tainted" ourselves and unconsciously start writing code which is based too closely on Borland/CodeGear code.

As everyone knows, we nevertheless all try our best to avoid committing tainted code, see e.g. the comments attached to the following bug reports:
* http://www.freepascal.org/mantis/view.php?id=7680
* http://www.freepascal.org/mantis/view.php?id=8248
* http://www.freepascal.org/mantis/view.php?id=9770

And yet, from time to time it nevertheless happens anyway:
* http://www.freepascal.org/mantis/view.php?id=8661

So this is one area where you, our users, can make a big difference and help us in preventing this from happening in the future, as well as with notifying us as soon as possible when it does happen (or when you know it happened in the past). Most importantly: please ensure that you *never* submit patches based on code copyrighted by third parties.

Let one thing be clear, however: none of the code that you wrote yourself can be affected by this issue, as long as this code is not based on the tainted code (whereby "based" means "based on their source code"; it does not mean, a.o., deriving from a tainted class, or calling a tainted routine -- in these cases your own code is clean). Simply recompiling your program with an untainted version of FPC will automatically clear your programs of any taint as well.

We are in a constructive dialog with people from CodeGear over this issue, and will report back later on the repercussions as far as current and previous FPC releases are concerned.

For the FPC Team,

Michael.

PS: for those interested, here is the preliminary list of tainted routines we have identified. If you are aware of more cases, please let us know as soon as possible so we can evaluate them and put them in the queue for re-implementation!

Classes:
* ExtractStrings
* TParser.TokenComponentIdent
* TParser.NextToken
* TReader.DoFixupReferences
* TReader.ReadCollection
* TReader.FindComponentClass
* TWriter.WriteComponent
* TWriter.WriteComponentData
* TWriter.WriteProperties
* TPropFixup.MakeGlobalReference
* FindNestedComponent
* GlobalFixupReferences
* GetFixupInstanceNames
* RemoveFixupReferences (RemoveFixups in Delphi),
* RedirectFixupReferences
* GetFixupInstanceNames (GetFixupReferenceNames in Delphi),
* RemoveGlobalFixup
* RemoveFixups
* TStringList.Grow
* TStringList.QuickSort
* TStringList.Find
* TCollection.GetNamePath
* TFPList.Assign (Kylix: tlist.assign)
* TFPList.Delete (Kylix: tlist.delete)
* TList.Extract

Strutils:
* SearchBuf
* DecodeSoundexInt

Varutils:
* DoVarClearArray

Math:
* MomentSkewKurtosis:

Zstream:
* Needs complete re-implementation (was taken from the net...)

Db:
* TDataset.SetFieldValues
* TDataset.DataEvent
* TDataSet.CalculateFields,
* TDataset.EnableControls
* TDataLink.CalcFirstRecord
* TField.RefreshLookupList
* TField.CalcLookupValue

Comctrl:
* HandleSafeCallException

Delphi Compiler is written in C/C++ ?!? (Closed source)

VCL/RTL is written in Delphi ! (Open Source)

Bye,
Skybuck.

Op dinsdag 20-11-2007 om 01:33 uur [tijdzone +0100], schreef Skybuck Flying:

Delphi Compiler is written in C/C++ ?!? (Closed source) VCL/RTL is written in Delphi ! (Open Source)

No. The VCL is written in Object Pascal, yes. But it's not open source. Borland and CodeGear published the code, so that people can take a look at it, to see how it works. For documentational purpose. And you can change the code if you need to, but you still need a Delphi license if you want to do that.

So the VCl and RTL are closed source, you may not use that code, unless you have a license, then that license allows you a few things...

Joost

Hello,

Skybuck Flying wrote

> But it's not open source. I like to think of open source vs closed source as: Viewable / Non-Viewable.

... and _that_ is where you are going wrong.

"open source" is a licensing term in these contexts.

By making source code viewable does not grant one a license to copy it, in the same way that you can't photo copy and distribute a book.

An amazing number of people seem to be highly confused by this.

This is probably why projects like FreePascal get sideswiped by tainted code on a too frequent basis.

Tony

Skybuck Flying wrote

Last time I checked object pascal was really old and Delphi added new language features.

Maybe you are getting confused with Pascal (the original one) and Object Pascal and then in the last few years the Delphi language.

There was three versions of Object Pascal. Apple, Turbo Pascal and Delphi. Most people only know the latter. I think in Borland Delphi v7, Borland started referring to the Object Pascal language as the Delphi Language, but all prior versions of Borland Delphi it was referred to as Object Pascal.

http://en.wikipedia.org/wiki/Delphi_(programming_language)

Free Pascal's implementation of the language is close to Borland's Object Pascal. I would guess from v5 onwards.

Regards,
- Graeme -

_______________________________________________
fpGUI - a cross-platform Free Pascal GUI toolkit
http://opensoft.homeip.net/fpgui/

On browsing through the code of
$FPCdir/packages/fcl-db/src/dbase/dbf.pas I came across the following (line 20-22)

// If you got a compilation error here or asking for dsgnintf.pas, then
just add
// this file in your project:
// dsgnintf.pas in 'C: \Program
Files\Borland\Delphi5\Source\Toolsapi\dsgnintf.pas'

Should this be removed? Does it infringe copyright? Is it even useful? Regards - Chris

dbf is Delphi + FPC buildable. This remark is just something related to Delphi inter version compability. It's harmless.

No, this is not a copyright infringement; TDBF needs to compile with older Delphi versions, and in Delphi 5 certain functions were in dsgnintf.pas, a unit that should normally never end up in an end-user program.

Michael.

Sorry if I'm a little late to this discussion. I just wanted to let you know that I am an attorney and have handled copyright infringement cases in the US (most recently the SONY vs. Crain music piracy case). As a hobby programmer and FPC/Lazarus enthusiast, I would be more than happy to review, in my spare time and free of charge, any copyright infringement notices that the team receives. My current position forbids me from what's referred to as "the outside practice of law", which means that I can't actually represent the team or write letters on anyone's behalf or offer binding legal advice, but I *can* offer my informed opinion as a member of the mailing list and fpc community who happens to be an attorney familiar with copyright law in the US...

The FPC team has been recently made aware that a number of routines in the Classes unit are apparently based on code originally from Borland/CodeGear.

Did the team receive a notice of infringement from CodeGear, or was this just part of an internal review which was discovered by the team itself? If anyone received any communication from someone at CodeGear, please forward it to me so that I can understand their position a little better.

Hmmm, it seems from the context that the original submitter may have believed that the offending code here had been copied from the LCL originally, but I may be mistaken. Anyway, this brings me to point out two things that may not be obvious to someone unfamiliar with copyright:
1) If the code was in fact used in a commercial "closed source"-licensed work but that code had originally appeared in an "open source"-style licensed work, the later license cannot add or remove restrictions to the original code (it may only do so to new code appearing in the same work, the original code is unaffected).
2) New code can be based on copyrighted code, as long as there is sufficient originality in the new code. In other words, as long as the code is reimplemented and not simply used verbatim, it is perfectly fine to base new code on copyrighted code, it is even ok to follow the original algorithm step by step (unless the algorithm is patented, which is a completely separate issue entirely). The rule of thumb is: if it looks copied, it probably was. On the other hand, simply changing names used in the code is not enough, contrary to what some people seem to think (names only account for a very small percentage of the written code, so that changing all names used in a source file only changes perhaps 5% or less of the text.)

The fact that code may have been copyrighted by a third party doesn't matter, only the license they used matters. A GPL-licensed work is indeed copyrighted.

We are in a constructive dialog with people from CodeGear over this issue, and will report back later on the repercussions as far as current and previous FPC releases are concerned.

Who originated this discussion (did CodeGear or their attorneys contact you, or did you voluntarily inform them of the issue yourselves)? And is this discussion fairly informal so far, or has CodeGear indicated they have the intention of pursuing the issue further, perhaps legally?

The initial accusations were published on a Blog of some former Borland employee or afficionado. He contacted codegear, apparently, and then a codegear official (Alan Bauer) contacted some of the FPC team members by private email.

Prior to this email, but after the abovementioned discussion on the blog, we had decided on an audit ourselves, and this yielded a result of some 20-odd routines which are suspicious, and we decided to play it safe and replace them.

All but 3 of these routines have meanwhile been recoded using a clean-room approach. We are waiting for the last 3, after which a new release will be made.

As far as I can judge:
They count on us to play things fair, and let us handle it to our own judgement for the moment, but said that if we undertook no action, they would take the necessary steps to protect their IP. We have given them the initial result of our review (i.e. the list of suspicious routines), and I haven't heard from them since. As soon as we merge the clean-room implementation to the main branch, I will inform them that this has been done.

Michael.

On 15/01/2008, Michael Van Canneyt wrote:
-- 8< --

thanks for your explanation, i was curious what the progress on this was. are there similar infringements in the lcl that needs to be replaced, or has this not been reviewed?

Michael.

henry

The FPC team has not reviewed the LCL; Maybe the lazarus team has;
However, I have no news of this.

Michael.

The tool I used to check the FPC sources is in FPC svn under utils/ sim_pasc and should compile with any C compiler under any *nix (and possibly also under Windows). See the file HOWTO-FPC.txt for how to use it. It has no problem with e.g. checking many include files vs a single unit.

Jonas

Michael Van Canneyt wrote

The initial accusations were published on a Blog of some former Borland employee or afficionado. He contacted codegear, apparently, and then a codegear official (Alan Bauer) contacted some of the FPC team members by private email.

A google search turned up the blogger in question, and these two articles: http://www.stevetrefethen.com/blog/IWishTheDelphiCommandLineCompilerWereFree.aspx http://www.stevetrefethen.com/blog/CopyrightAndTheFreePascalProject.aspx

He worked at Borland for 15 years, and CodeGear the last couple of those, but works for a new company now with 4 other ex-CodeGear employees. Again, I'm late to the party, so this has probably already been mentioned before on the list, but from some of his comments it is apparent that he came to his conclusions as to the copyright issue at the beginning of last year while think it's
interesting that he never raised those issues then. It is also clear that he has no clue about copyright law. It seems he believes that if a unit does the same thing in the same way as another copyrighted unit, there is a copyright violation somewhere involved. You can't copyright an algorithm (and there's still an ongoing debate about whether it's actually proper to patent one, although the USPTO does it all the time now). But it doesn't help when someone is blowing smoke into an already cloudy issue.

Judging from the posts in this thread, it does seem that the FPC team found some code after their review that needed fixing, and I'm glad that's mostly been finished. However, it does concern me that the LCL team hasn't performed the same kind of in-house analysis on their code yet. Until they do, I'm afraid this issue won't go away anytime soon.

Please forgive my ignorance, but what is LCL?

Mark Emerson

>yet. Until they do, I'm afraid this issue won't go away anytime soon. Please forgive my ignorance, but what is LCL?

The visual classes libray of Lazarus.

John Stoneham ha scritto:

I am pleased to read your competent opinion about this matter that I consider very interesting and relevant (both the opinion and the matter).

Could you please tell me if there is a real chance that CodeGear asks money for a concrete copyright violation to someone of the FPC or Lazarus developers?

And in this case, how this claim could be performed?

thanks
tiziano

Well, one interesting thing to note is that I do not know of a case where a small code segment was sufficient to bring a copyright infringement action, even if that code segment was a couple of hundred lines long. However, this may be due to two factors: 1) such a relatively small code segment is fairly easy to re-implement as original code; and 2) this usually reflects only a very small fraction of the entire codebase involved.

Point 2 is worth commenting on. This would be the equivalent of a songwriter using a 4-note phrase from another song. US Copyright law has recognized the impracticality (and indeed, undesirability) of such a strict policy if it were to be enforced. The fact that a several-hundred-thousand line codebase has even a few hundred lines in common with another work is rather unremarkable from a legal standpoint. However, that's not to say the plaintiff couldn't bring such a suit; but filing a suit based on code fragment violations in software is a fairly new concept, and it's largely been unsuccessful. More frequently, software "infringement" suits are brought as patent claims, not copyright, specifically because of the difficulty in enforcing a copyright claim when only a small segment of code is alleged to have been copied. (Copyright infringement claims usually involve the copying of entire works, not tiny segments, and indeed the Copyright law itself was written with that idea in mind.)

Copyright law is also fairly unique in that it provides for statutory damages. What this means is that the plaintiff only has to prove that an infringement occurred, they don't have to prove that they were actually damaged by that infringement. Usually, in a civil suit the plaintiff has to prove that they were actually damaged in some way, and the judgment has to reflect the actual damages incurred. As opposed to proving actual damages, US copyright law provides the plaintiff with the option of requesting an award of damages which is set by the statute itself, which is why this type of award is called "statutory damages". The range is a fairly large, and it is in the judge's (or jury's) discretion to set the actual amount, but the award has to be in the range of $750 to $30,000 per work infringed. One software program is considered one work, so it wouldn't matter is there were several instances of copyright infringement involved within the same program, there would still only be one work infringed upon and only one award of damages (using FPC as an example, CodeGear couldn't ask for (say) 30 separate damage awards if 30 units within the FPC codebase contained infringing code, they could only request one award of damages since FPC is considered one work). There is a second, higher tier of damages, if the plaintiff can prove the infringement was "willful", set at $150,000 per work. However, this is very hard to prove. "Willful" has a legal meaning in this context, it doesn't simply mean that the defendant *intended* to infringe upon the copyright, which is almost always the case anyway. An example of "willful" infringement would be a CD-pressing shop. If you burn a copy of your friend's CD, you most certainly intended to infringe the copyright. But the intent would only be considered "willful" if you burned (say) 1,000 copies with the intention of selling them (actually, even if you intended to just give them away -- profit is not required).

Tiziano De Togni wrote

Could you please tell me if there is a real chance that CodeGear asks money for a concrete copyright violation to someone of the FPC or Lazarus developers?

FPC developers approached them (CodeGear) and took it on ourselves to solve the issue. We found a automated tool to help, listed the suspect code and fixed it by rewriting the suspect code. CodeGear has nothing to complain about as far as I can see! FPC developers did not wait to be reprimanded - they fixed it as soon as they found out about it.

Regards,
- Graeme -

_______________________________________________
fpGUI - a cross-platform Free Pascal GUI toolkit
http://opensoft.homeip.net/fpgui/

I've been following this topic from the beginning and I took the time to read the (questionable) blog from the ex-CG developer.

I'm interested in it because aside my personal enjoyment, I started using freepascal+lazarus where I work in order to quickly solve some needs like data analysis, algorithms benchmarking before implementation, controlling our hardware from the pc, and so on...related to the hardware we design (I'm a hardware engineer).

Well, what kind of implication should I expect if, for any reason, things turn to bad?
Those implication suggested by the blogger are so serious that if I were trusting him, I would immediately erase all disks with my pascal stuff and buy new ones.

R#

2008/1/16, Graeme Geldenhuys :

Not too much implication, for you, normally:

1. If you have a licensed copy of Delphi, you can ignore it: you are allowed to use this source code.

(We do not have the right to distribute it, but that is our problem, which we must solve, and which we are solving)

2. If you don't have a licensed copy, you can still use the GPLed version of CLX which is freely downloadable from Sourceforge: it is the same code as the Delphi code. It does mean that your application needs to be GPLed. If you don't distribute them, this is not a problem at all...

So the only case when you could have problems is when both the following conditions are true:
a) you don't have a registered Delphi
b) you use FPC/Lazarus to create and distribute closed-source commercial software.

In this case, then the remedy is to wait for the next release which has the clean-room code, and simply recompile your application. It is also under the assumption that Codegear actually can prove that we have been infringing on copyright.

Things get more complicated even because Codegear is US based, and most FPC development happens in Europe and south america: I don't know whether a US copyright is enforceable here in europe.

Note that this is my understanding of things, and in no way binding legal advice, I am not a lawyer after all... :-)

Michael.

Thanks a lot!
you can't find a lawyer anywhere when you need it! :-) I'm joking, of course...

Given that I don't have Delphi, suppose that company X ask me to make a software for them. I might give them the software, with full source code and a GPL licence note every here and there, and ask money for the _design_ of the software, instead of the software itself. How does it sound to you and eventual hidden lawyer "listening" to us ?

thanks for your job, really!
If I were even a little bit good at programming (serious programming and not just having a computer do what I need), I would be the perfect clean room programmer....never had any sort of delphi and never read any source at all !

cheers,

Roberto

2008/1/16, Michael Van Canneyt :

Actually, none of this is necessary if the "questionable code" is part of a library routine which is unused and unlinked by the developer, since the resulting program will not contain the code in question. However, if it's part of the compiler itself and not part of the runtime, that raises another question (this whole issue is getting rather interesting from a legal standpoint). Let's say that Corel has infringed upon Microsoft's copyright by including some code from MS Word in WordPerfect. Clearly, Corel is the infringing party. But what does that mean for a user of WordPerfect? Does that mean an author working on a novel is also violating Microsoft's copyright by using WordPerfect? I think almost everyone would agree that he's not. But what if he *knows* that the version of WordPerfect he's using contains copyright violations? Now it starts to get a little murkier, but I think the answer is still "no".

A compiler is not very different conceptually from a word processor. Should the result be any different? Again, I don't think so. If the word processor, when generating a document, injects into it some predefined text supplied with the program as text snippets, and those snippets contain text which is actually copyrighted by someone else, then the author needs to make sure those infringing snippets are not included in the completed work he distributes. The same analogy applies here.

It is also under the assumption that Codegear actually can prove that we have been infringing on copyright.

Here's an interesting question: Is any of the code at issue also in CLX? If so, I would strongly argue that there is no infringement possible since Borland GPLed that code over half a decade ago (it doesn't matter legally that they may have also included it in the closed-licensed VCL -- once it's GPLed, it's GPLed for good).

Things get more complicated even because Codegear is US based, and most FPC development happens in Europe and south america: I don't know whether a US copyright is enforceable here in europe.

The short answer is yes, there are treaties that make it so.

But here are the most relevant issues I see:
1) Code fragments representing a small fraction of a codebase would rarely be sufficient enough to establish that a work had been infringed.
2) The fact that some of these code fragments may also be contained in the CLX would in itself invalidate any potential claims of infringement (this really needs to be looked into).
3) The average user of FPC does not need to be worried about this affecting their own code (unless their own code happens to include some of the offending code, and even then see #1 above).
4) The biggest concern I see is not the potential copyright penalties, but the negative publicity and reception in the open source and developer community.

Note that I would be drawing different conclusions if the allegations were involving a substantial portion of the codebase instead of just fragments.

Here's an interesting question: Is any of the code at issue also in CLX?

Yes. To my knowledge all affected code is GPLed. However the FPC RTL is modified LGPL which is GPL incompatible in this regard (you might not change GPL licenses into LPGL ones).

In each case, the 'suspect' code is not used in the compiler at all.

The suspect routines are mostly in the classes unit. In the component streaming system, to be exact. This code is of course available in the GPL-ed CLX... Note also that the CLX is under some kind of dual license: GPL or the Borland license. Which license is used depends on the software engineer, if I understand correctly.

If someone takes this point of view:
That would effectively mean that any application built with it is also GPL-ed. The Free Pascal license explicitly also allows to create closed-source applications.

Regardless of this, we do not wish to be suspected of using Borland's code, and therefor decided to play it safe and recode the routines anyway. I cannot stress this enough.

Michael.

Michael Van Canneyt wrote

Regardless of this, we do not wish to be suspected of using Borland's code, and therefor decided to play it safe and recode the routines anyway. I cannot stress this enough.

Are the recoded routines in 2.2.1?

Any vague ideas when 2.2.2 would be coming out? =)

thanks,

Roberto Padovani wrote

Given that I don't have Delphi, suppose that company X ask me to make a software for them. I might give them the software, with full source code and a GPL licence note every here and there, and ask money for the _design_ of the software, instead of the software itself.

Another misunderstanding of the GPL. Of course you are allowed to charge money for your software.

But given it is GPL your customer may decide to redistribute the source, as you gave him explicit permission with the GPL to do so. It would simply put all your other code under GPL, too.

But there's absolutely *nothing* in the GPL that prevents you from charging money for your work.

Vinzent.

And that is yet another misunderstanding of the GPL: it would *not* automatically make all the other code of that program GPL.

If you do not license the rest of the source code under the GPL (or under a GPL-compatible license), you are simply infringing on the copyright of the author(s) of the GPL code: your are using that code in a way which is not allowed by the GPL. You can solve this in two ways a) license the rest of the source code of that program under the GPL or a GPL-compatible license
b) stop distributing the program containing GPL code (and possibly replace the GPL code with otherwise licensed code, and then distribute the rewritten program)

In any case, whoever receives a program containing a mixture of GPL and GPL-incompatible code, is not allowed to further distribute it either (because he would infringe on the copyright of author(s) of the GPL code just as much as the initial distributor).

Jonas

Well, well, I didn't say *that*. But with "and a GPL licence note every here and there" I would have assumed it, of course.

If you do not license the rest of the source code under the GPL (or under a GPL-compatible license), you are simply infringing on the copyright of the author(s) of the GPL code: your are using that code in a way which is not allowed by the GPL.

Ok, yes. If you read it this way. OTOH it said "with full source code", so as a customer I'd "assume" at least, that everything is GPLed here, even if it does not say so explicitely in each single source file.

And even then, you would be allowed to give it to the customer as is, as you're providing the full source code as required by the GPL.

In any case, whoever receives a program containing a mixture of GPL and GPL-incompatible code, is not allowed to further distribute it either (because he would infringe on the copyright of author(s) of the GPL code just as much as the initial distributor).

Sorry, but that's not true per se. I *am* allowed to use GPL code in an internal project without redistributing it.

Ok, the next paragraph can be considered quite cynical, so if you don't like, don't read it. AFAICS I am allowed to use GPL code in a bomb's flight control program and I am certainly not required to distribute the source code to the enemy together with the bomb, am I?

Vinzent.

Sorry, that's how I understood "It would simply put all your other code under GPL, too.".

The GPL requires more than full source code, it also requires that the customer has the same rights to make modifications and redistribute everything under the GPL/GPL-compatible license(s). So no, simply giving the full source code to the customer is not enough: everything must also be licensed under a GPL/GPL-compatible license.

Yes, that's correct. But then you're no longer talking about customers.

Jonas

I took a look at Borland's license distributed with Kylix. That is one bizarre creature. They go through all kinds of contortions to make it "dual licensed". I don't think the GPL part qualifies as a GPL license at all, actually. I think they felt backed into a corner by having to use Qt on linux and had to include some kind of GPL license because of that but at the same time wanted to keep as much control as they could. Anyway, the "dual license" itself is kind of irrelevant since each source file in the CLX contains it's own copyright notice and specifically states "This file may be distributed and/or modified under the terms of the GNU General Public License (GPL) version 2 as published by the Free Software Foundation". Even if Kylix contained a more convoluted license in general, since each source file contains that notice, the GPLv2 covers the source in each file that notice is in.

Anyway, it's all a fairly moot point now (or soon will be).

Regardless of this, we do not wish to be suspected of using Borland's code, and therefor decided to play it safe and recode the routines anyway. I cannot stress this enough.

I agree completely with this. As an attorney, I like to make arguments and debate things and explore nooks and crannies. But I agree that it is best to have completely "pure" code in FPC, so that no Borland/CodeGear fanboy can make any claim, unfounded or not, about FPC "stealing" from CodeGear.