1.

I am trying out the Lazarus demos in tiOPF and I have come across an error.

When I compile Demo_OneToMany I get the error:

FMainOneToMany.pas(90,20) Error: identifier idents no member "AddField"

lTableMetaData.AddField('OID', qfkString, 36); // Using GUID OIDs

I fixed this by changing to AddInstance as it was commented out and documented in the source.

2. Is the source for the lazdatadesktop available.

Op 2011-03-14 13:17, Frank Church het geskryf:

I fixed this by changing to AddInstance as it was commented out and documented in the source.

Thank for the note, I'll update the demos.

2. Is the source for the lazdatadesktop available.

Yes, it's included with Lazarus in the <laz>/tools/lazdatadesktop/ directory. For convenience (mainly for Delphi developers), I have included binaries of Lazarus Data Desktop at the tiOPF SourceForge Download page. The binaries is a bit old I think.

Regards,
- Graeme -

Issue 3:

Is there a non GUI example that stores the data to a SQL table, and can create it as well?

In Demo_OneToMany I tried to test SQLite by changing these two lines:

Demo_OneToMany.dpr (After Application.Initialize)

  //gTIOPFManager.DefaultPersistenceLayerName := cTIPersistXMLLight;
  gTIOPFManager.DefaultPersistenceLayerName := cTIPersistSqldbSQLLite3;

FMainOneToMany.pas

  // Database file name
  //cDatabase = 'testdb.xmll';
  cDatabase = 'testdb';

After I make this change, a SQLite table is not created, but instead a directory is created and CSV files are created there. In the Compiler options of tioXXX I ensured that -dLINK_SQLDB_SQLLite3 is present. Is some kind of tiQuerySQLLite required that is not yet implemented? I can't see a reference to SQLite in the tiOPF uses list.

Issue 4:

When I try to compile tiOPFGUI it halts with this error:

C:\lazarus28240\components\tiOPF2\Source\GUI\tiPerAwareCtrls.pas(157,28) Error: No property found to override

The property in question is TtiPerAwareAbs.ParentCtl3d in unit
tiPerAwareCtrls;

From what I understand from the docs the custom controls that come with it

will not work in the IDE, but shouldn't they be able to work if they are createed at runtime?

I want to get acquainted with the non GUI aspects, such as creating objects and saving them, then turn to Model-Gui-Mediators later on,

Op 2011-03-14 19:14, Frank Church het geskryf:

If you only use the "core" units (and some units from Options directory) located in tiOPF.lpk, then they are all non-GUI. As for the demos, I think they are all "gui" demos simply to make it more visually pleasing to the end-user/developer.

The whole framework is unit tested, so you could look at that code too, if you want more examples of using the framework. The testing framework is non-gui too.

That should be all that is required, but it seems the CSV persistence layer is still taking priority in our code, for some reason. Maybe place 'x' characters behind all the persistence layers you don't want - thus only include SQLDB_SQLLite. That's in tiOPF.lpk | Compiler Options | Other dialog. See what happens then.

I must add, the SqlDB persistence layer was written by Michael to support all backends that SqlDB (in FPC) supports, but when I tested Postgres via SQLDB I found lots of issues. The tiOPF's SqlDB persistence layer seems to need quite a bit more refactoring to work correctly with all SqlDB backends. The only one really tested at the moment is SqlDB+Firebird, as that is what I (and I believe Michael and others too) uses in our commercial apps.

If I can setup a SqlLite server or something, then I can start testing that persistence layer under tiOPF too. I'll see what I can manage.

In the Compiler options of tioXXX I ensured that -dLINK_SQLDB_SQLLite3 is present. Is some kind of tiQuerySQLLite required that is not yet implemented? I can't see a reference to SQLite in the tiOPF uses list.

The only reference to the various persistence layers should be in the uses clause in the implementation section of tiOPFManager.pas unit. The rest are all controlled by the -dLINK_xxx compiler defines.

I know nothing about SQLLite.... Is it free to use? ie, can I download something like a client library or server (that's if everything isn't included with FPC itself), and include it in my hourly build sever runs? I already test 5 persistence layers like that using FPC 32bit and 64bit... I should really include SqlLite, MySQL and Postgres in those runs too - at least I know the latter two are free, plus we have a spare linux server here, that is not doing much at the moment.

It's probably a small thing to fix (add yet another IFDEF), but I gave up trying to maintain the LCL & VCL custom components of tiOPF. LCL is just too different, causing lots of code obfuscation due to IFDEF's, or the Lazarus IDE Forms Designer was too buggy to justify my time. That's why I implemented and started using Model-GUI-Mediator (MGM) instead.

I highly recommend you use MGM for any tiOPF GUI work. It's very versatile, it is unit tested (mostly), and is used by a few developers in commercial environments, so proves to work well.

From what I understand from the docs the custom controls that come with it will not work in the IDE, but shouldn't they be able to work if they are createed at runtime?

That will eliminate some of the Lazarus problems yes, but you still have the very obfuscated GUI code to work through if anything goes wrong. More and more IFDEF's seem to be required to get the two (LCL and VCL) to work. Lazarus LCL was designed to convert Delphi code to Lazarus, not to have shared-code components.

LCL is not alone, I had the exact same problem with Kylix's CLX and Delphi's VCL in the tiOPF GUI components.

I want to get acquainted with the non GUI aspects, such as creating objects and saving them, then turn to Model-Gui-Mediators later on,

OK, creating some non-gui demos will be easy to do... plus the code should work as-is with both FPC and Delphi. I'll see what I can put together. I wouldn't discard the "gui" demos though, the code inside OnClick even handlers should be the same for gui or non-gui apps.

For any further tiOPF specific help, can you please move your discussions to the dedicated tiOPF support channels. This will then benefit other future tiOPF users too. There is a tiOPF NNTP news server (see the website for details), or the tiOPF mailing list (see the SourceForge project page for details).

Regards,
- Graeme -

On 15/03/2011 08:57, Graeme Geldenhuys wrote:
[...]

SQLite is (or at least I think it is) the library that makes your program think it talks to fully loaded database server, where in fact it is just a file or two; has the advantages of fully database operation (speed) and can be set up to work fully in memory, but without special setup (I think) it can't handle concurrency;
(Firefox uses that a lot, currently actually for everything : ever wondered how they implemented Private Browsing vel P**n Mode ?)

L.

Graeme Geldenhuys wrote

If I can setup a SqlLite server or something, then I can start testing that persistence layer under tiOPF too. I'll see what I can manage.

No need to setup a server. Just download the library (or install it using your distro's package manager) and be happy.

SQLite is licensed as Public Domain and only consists of a small (~300kB) library that contains the whole DB engine. It also supports pure in-memory databases ;)

Thus its best suited for embedded cases (we use it on Windows Mobile devices).

For more information see here: http://www.sqlite.org/

Regards,
Sven

Op 2011-03-15 12:13, Sven Barth het geskryf:

No need to setup a server. Just download the library (or install it using your distro's package manager) and be happy.

Thanks.

OK, so it's like Firebird Embedded DB's then. That's nice and easy. :)

Thanks for the information.

Regards,
- Graeme -

I don't recommend it for use with pascal, since it does no type checking whatsoever - at least last time I tried - thus you can perfectly store a string in e.g. an integer field.

Combined with Pascal code, this can lead to very unfortunate results.

Michael.

Op 2011-03-15 14:36, het geskryf:

Do you mean the "no type checking" issue is with Firebird Embedded, or with SQLLite?

Regards,
- Graeme -

On Tue, 15 Mar 2011, Graeme Geldenhuys wrote:

[...]

Whaddya want, it's a c[++] library...
they like shooting themselves in a foot.

Is there a Pascal wrapper available that _would_ do the above ?

L.

SQLLite obviously. Firebird embedded is a full firebird, with all functionalies of a server.

Michael.

If you only use the pascal code to access the sqlite library, it should be safe. If other libs access it as well, I would not recommend it.

Michael.

Op 2011-03-15 15:04, het geskryf:

SQLLite obviously. Firebird embedded is a full firebird, with all functionalies of a server.

That what I thought. But you had me worried there for a second! :)

By the way, that SQLLite issue seems very similar to the one in MySQL too. The latter allows you to setup field constraints without problems (yeah, they finally caught up on that one), but when you insert data, such field constrains are simply ignored. :-)

It still blows my mind why Firebird RDBMS isn't more popular, yet half-baked ones like MySQL (and from what I can read SQLLite too) is.

Regards,
- Graeme -

Simple: Sloppy work is a virtue these days.

Michael.

While the type-safety can potentially lead to problems when non-Pascal systems access the database as well, I wouldn't describe SQLite (btw: only one "L" ;) ) as half-baked. It was a decision of the developers to allow this (although one might argue how they came to such a decision...).

Nevertheless SQLite has its usage, because it is so small (to be fair: I don't know the size of the Firebird embedded lib), so it's suited very well for embedded applications. This (and the license) might be the reason, that SQLite is used in iOS and Android as well.

Regards,
Sven

I use both, so I will try to explain why I chose to use SQLite in some projects.
- SQLite is faster, MUCH faster, absolutely no comparison.
- SQLite is easier to use. It's especially nice to be able to deploy your app and easily create the SQLite is made for SINGLE application access. So my pascal program will always read/write the expected types.
- Firebird embedded does not work in virtual machines when the DB is on a network mapped drive. I have tested VMWare, Virtual Box and Virtual PC. (This can also be a plus since I use this fact to inhibit one of my apps to be run from a virtual box.)
- SQLite can be embedded without needing to distribute a DLL. (Although I no longer use this feature)

I only use Firebird in one app so the user can create his own reports. Firebird is a fantastic DB-engine, but has way too much for most of my needs. SQLite is more perfectly suited for me.

I know not everyone will agree, but I am an SQLite fan.

Andreas

Op 2011-03-15 15:49, Sven Barth het geskryf:

Which such decisions, I'll always call them half-backed. I was trying to find the original MySQL article I was reading the other day that show that HUGE flaw with data integrity, then I came across another website listing about 20 other huge flaws of MySQL. Some where quite ridiculous, like inserting data, but when reading that data back, the values have changed because of overflow issues etc.. Your data has NO guaranteed to be safe in MySQL - probably why it is so popular for toy projects like websites [where data clearly has no great valuable].

As I said, I don't know SQLite, but if they do similar stupid things like MySQL, then they are no good to me either.

Nevertheless SQLite has its usage, because it is so small (to be fair: I don't know the size of the Firebird embedded lib), so it's suited very

Firebird Embedded uses a +-200KB lib (if I remember correctly). That's pretty small, and it can be shared between many apps because it's not compiled into each executable.

Firebird (and it's parent Interbase) has been around for year, is proven technology; has rather big clients like the US Military etc. We have put Firebird though it's paces (like power interruptions, broken network links etc), and the data always came out reliable.

Regards,
- Graeme -

Op 2011-03-15 16:05, Andreas Berger het geskryf:

- SQLite is faster, MUCH faster, absolutely no comparison.

Honest question. Do you actually have some speed comparisons between Firebird and other database, including SQLite? I would really like to see those please.

- SQLite is easier to use. It's especially nice to be able to deploy your app and easily create the do this too (deployment and run-time creation) with Firebird - no

problems at all.

- SQLite is made for SINGLE application access. So my pascal program will always read/write the expected types.

The same database application can easily switch between Firebird Embedded and Firebird RDBMS - thus you have a great upgrade path to a full blown multi-user system, if you application becomes more popular and needs to scale upwards.

- Firebird embedded does not work in virtual machines when the DB is on a network mapped drive. I have tested VMWare, Virtual Box and Virtual PC. (This can also be a plus since I use this fact to inhibit one of my apps to be run from a virtual box.)

I can't comment on this, but I can't see why not, if you for example under Windows map the network share to a permanent drive letter. We have done this with MS Access (.mdb database) for years. And MS Access is a local database only system too. The other point being, if you database needs to live on a remote location, why not simply switch to Firebird RDBMS server - get the full multi-user treatment for free.

- SQLite can be embedded without needing to distribute a DLL. (Although I no longer use this feature)

... and Firebird Embedded can share it's dll with multiple applications on that same system, reducing the byte size of each database application on that system. ;-)

Firebird is a fantastic DB-engine, but has way too much for most of my needs.

I'd rather go with too much features (though I don't see why this is a problem, just don't use those features), and a database that keeps your data (integrity) in check.

PS:
I would honestly like to see the speed comparisons you have with Firebird. I have never seen good comparisons, and the ones I have done myself, like comparing Firebird to Postgress - using the exact same application, Firebird came out tops (by far). But then again, I have no tweaking/admin experience with Postgress - I simply compared speed with default installs of each RDBMS in question. Apparently Postgress needs lots of tweaking before it performs well. :-/

Regards,
- Graeme -

This is contrary to what my tests show. For simple queries: yes, sqlite is slightly faster.

For slightly complex to complex queries: no way. As soon as 2 or more joins enter the query, SQLite was beaten by Firebird hands down.

I tested using a database with >600.000 records - it makes no sense to do
speed comparisons on small databases.

Michael.

I can't comment on the other points, but on this one: You can do this with SQLite as well. It's just a normal shared library (that can be statically linked if you want, of course).

I have another point for SQLite:
SQLite supports more (processor) platforms than Firebird. Especially it seems that Firebird does not support ARM - please correct me if I've read this wrong on the Firebird site.

Regards,
Sven

I've actually enjoyed this thread up to now, but it's getting up to the flamebait and fud stage. Facebook primarily uses MySQL, as do many other of your "toy" applications. Perhaps this kind of ranting can be reserved for your blog?

Henry

Firebird developers discovered long ago that file integrity is difficult to maintain when multiple programs access files across network connections and a wide variety of networking clients are used. Therefore, to avoid random issues that would be mistakenly attributed to Firebird, and especially to avoid corrupted databases resulting from the networking layer's failures, they have long prohibited Firebird databases from being located in remote locations.

However, the most recent versions allow for an override in the Firebird config file so you can actually do this as long as you take FULL responsibility for the results.

Hi Graeme,

I'm not trying to say that SQLite is better than Firebird. In fact I use FB for larger database usage. What I was trying to say is that SQLite has it's place and worth. I replied due to you thinking that SQLite was half-baked and Michael saying that it is sloppy. I'm not trying to flame, only present my view since I know that SQLite could be usefull to pp out there.

Now to the issue of speed. We had a program that was written in DOS that kept a record of all occurances in the customers equipment. I used a homemade DB that recorded around 20K records per day. Last year we upgraded part of the program to Windows and the DB was part of the upgrade. I used the program started the first
time at a client, it would convert the last 24 months worth of data. It was taking around 80 minutes to convert one month. Before the project was finished a decision was made to use SQLite. After converting we discovered that the conversion of the DB took only around 50 minutes with SQLite. I have made no query tests. This was purely writing to the DB. By the way, the DB has 6 tables and 4 of the tables 8 indexes. Querying is and was blazing fast in both DB's, I never compared.

managed to create the DB. Maybe it has to do with Zeos that throws an exception when I enable it without having a valid firbird DB. But I would like to know how to do this

True, and for this and many other things firebird is far superior. But often I don't have the need for this.

Already answered by someone else. Rats, there goes my virtual machine test.

Data integrity is not an issue. SQLite has no problems there. They are simply not orthodox and you can (if you want to) access a number field as text or try to access a text field as an integer. But you don't have to.

Andreas

Op 2011-03-15 17:38, Rich Saunders het geskryf:

Firebird developers discovered long ago that file integrity is difficult to maintain when multiple programs access files across network connections and a wide variety of networking clients are used.

I fully agree with this. And it applies to any "single user" database file. Firebird, Sqlite, MS-Access, csv, tab, ini, TDBF etc...

Therefore, to avoid random issues that would be mistakenly attributed to

A valid enough reason, especially considering that they do offer a full blown SQL database server too - with no extra charge.

Firebird, and especially to avoid corrupted databases resulting from the networking layer's failures, they have long prohibited Firebird databases from being located in remote locations.

The exact same things applies to MS-Access, and yes we have such installs in over 1000 clients and have to live with the corrupt database issues for a little while longer [we already suffered for 12 years with this]... The new version of our application now uses a Firebird DB server instead. Thank goodness! :)

But just like MS-Access, I am pretty sure you can "fool" Firebird Embedded by mapping a share to a drive letter (if you are under Windows), or mount an NFS share as part of your local filesystem (if you are under Linux), and Firebird Embedded will not be the wiser.... But yes, you still have to live with the limitations included in any single user database file.

Regards,
- Graeme -

Op 2011-03-15 17:32, Henry Vermaak het geskryf:

I've actually enjoyed this thread up to now, but it's getting up to the flamebait and fud stage. Facebook primarily uses MySQL, as do many other of your "toy" applications.

As far as I know (I read it recently - I'll look for the link) Facebook uses a custom written database, not MySQL. Same goes for Google's Gmail. It's not even a relational database, but some "new" hybrid database type optimized for there specific data.

Regards,
- Graeme -

Op 2011-03-15 17:28, Sven Barth het geskryf:

I can't comment on the other points, but on this one: You can do this with SQLite as well. It's just a normal shared library

I knew that, that is why I include the smiley. I meant that statement as a joke.

Regards,
- Graeme -

In that case I have misinterpreted your smiley. ^^ I'm sorry.

Regards,
Sven

http://royal.pingdom.com/2010/06/18/the-software-behind-facebook/

Google AdWords uses MySQL, so does Wikipedia.

Henry

Op 2011-03-16 11:09, Henry Vermaak het geskryf:

http://royal.pingdom.com/2010/06/18/the-software-behind-facebook/ Google AdWords uses MySQL, so does Wikipedia.

Thanks for the link, some interesting items listed there.

Then again, there are so many conflicting links on the internet. Even many of the references listed on the MySQL wikipedia page doesn't describe/reference the claims they make. :-)

That's the internet for you! Even a 5-year old could be writing those web pages - we would never know. :)

Anyway, there are some ex-Google employees that mention that Google Ads used to use MySQL, then switched to something proprietary (alas with initial speed problems), so now it is unclear what Google actually uses for Google Ads.

http://zurlocker.typepad.com/theopenforce/2005/12/googles_use_of_.html http://xooglers.blogspot.com/

I'm still searching for the link regarding the DB backend for Google's Gmail and lots of Facebook content storage. I remember being very interested in this "new database engine" they talked about, but remembered that they did mentioned it was only good for certain types of data.

Regards,
- Graeme -

Henry Vermaak wrote

I've actually enjoyed this thread up to now, but it's getting up to the flamebait and fud stage. Facebook primarily uses MySQL, as do many other of your "toy" applications.

(one should make a difference between using a DB mainly as storage layer, and a full transactionally-safe RDBMS. MySQL is traditionally mostly used for the former)

Op 2011-03-16 19:20, Marco van de Voort het geskryf:

(one should make a difference between using a DB mainly as storage layer, and a full transactionally-safe RDBMS. MySQL is traditionally mostly used for the former)

True. I can also argue that any project that calls themselves a "database", or "database server", or RDBMS, should not just guarantee a storage layer, but also what you put into the storage layer is what you get out of the storage layer, plus the base line "relational database" features one would expect from todays software.

MySQL fails on many of those points, and definitely has the issue of 'what you put in is not guaranteed what you get out'. I must admit, MySQL 5.x has greatly improved in functionality and reliability of data
- but it doesn't come close to matching Firebird, Postgress, DB2, MS-SQL Server, Oracle etc in features and data reliability.

I think the only positive point for MySQL is that for some insane reason it got associated/included with the toolset "LAMP". To many management type people only know the keywords or acronyms of the month, but not really the technology behind them - thus they often make the wrong decisions when it comes to what software to use. That's life I guess.

Regards,
- Graeme -

Graeme Geldenhuys wrote

Do you mean the "no type checking" issue is with Firebird Embedded, or with SQLLite?

Just a side note:
It's just "SQLite", with only one "L". You might also want to fix this in the tiOPF related file names.

Andreas Schneider wrote

It's just "SQLite", with only one "L". You might also want to fix this in the tiOPF related file names.

Thanks for that. I fixed the related units in tiOPF2 and tiOPF3 as well.