Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-devel / 2012-04

sqlite support for Android

6 messages · 3 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Felipe Monteiro de Carvalho

Hello,

In Android a dlopen operation without a full path will fail:

  sqliteDLL:=DlOpen('libsqlite.so',RTLD_LAZY); ==> fails
  sqliteDLL:=DlOpen('libsqlite3.so',RTLD_LAZY); ==> fails
  sqliteDLL:=DlOpen('/data/data/com.pascal.lcltest/lib/libsqlite3.so',RTLD_LAZY);
 ==> fails
  sqliteDLL:=DlOpen('/data/data/com.pascal.lcltest/lib/libsqlite.so',RTLD_LAZY);
 ==> works
  sqliteDLL:=DlOpen('/system/lib/libsqlite.so',RTLD_LAZY); ==> works

So the existing sqlite bindings fail. The simples solution would be hardcode for Android /system/lib/libsqlite.so but this is somewhat of a grey area since this library is not guaranteed to exist. The other option is placing it in our lib folder, but then you need to pass the entire path for dlopen and this path will necessarely change with each project, for example: '/data/data/com.pascal.lcltest/lib/libsqlite.so' constains the name of the project "com.pascal.lcltest"

The solution would be to make this library name a variable then, but this won't work because the sqlite bindings need to support static linking too =O So maybe a variable only in the case of dynamic binding.

Well, so basically I am just sending this e-mail to start talks about what kind of solution would be accepted, maybe someone has better ideas then I had. I am tempted towards:

{$ifdef Android}
  libsqlite3 = '/system/lib/libsqlite.so';
{$else}
...

Which is what I already commited to fpc4android

Felipe Monteiro de Carvalho

Reply by email

Unknown author
to Felipe Monteiro de Carvalho
Felipe Monteiro de Carvalho wrote 25 lines
Hello, In Android a dlopen operation without a full path will fail: sqliteDLL:=DlOpen('libsqlite.so',RTLD_LAZY); ==> fails sqliteDLL:=DlOpen('libsqlite3.so',RTLD_LAZY); ==> fails sqliteDLL:=DlOpen('/data/data/com.pascal.lcltest/lib/libsqlite3.so',RTLD_LAZY); ==> fails sqliteDLL:=DlOpen('/data/data/com.pascal.lcltest/lib/libsqlite.so',RTLD_LAZY); ==> works sqliteDLL:=DlOpen('/system/lib/libsqlite.so',RTLD_LAZY); ==> works So the existing sqlite bindings fail. The simples solution would be hardcode for Android /system/lib/libsqlite.so but this is somewhat of a grey area since this library is not guaranteed to exist. The other option is placing it in our lib folder, but then you need to pass the entire path for dlopen and this path will necessarely change with each project, for example: '/data/data/com.pascal.lcltest/lib/libsqlite.so' constains the name of the project "com.pascal.lcltest" The solution would be to make this library name a variable then, but this won't work because the sqlite bindings need to support static linking too =O So maybe a variable only in the case of dynamic binding.

This variable already exists: SQLiteDefaultLibrary.

Show quoted text 7 lines
Well, so basically I am just sending this e-mail to start talks about what kind of solution would be accepted, maybe someone has better ideas then I had. I am tempted towards: {$ifdef Android} libsqlite3 = '/system/lib/libsqlite.so'; {$else}

This is the most practical solution, just redefine the constant.

The SQLiteDefaultLibrary variable is initialized from this constant.

In case a private library is used, users can always explicitly do a

InitialiseSQLite('/path/to/private/sqlite.so');

to override the default behaviour.

Michael.

Reply by email

Jonas Maebe
to Felipe Monteiro de Carvalho: In Android a dlopen operation without a full path will fail:…

Felipe Monteiro de Carvalho wrote on Mon, 02 Apr 2012:

In Android a dlopen operation without a full path will fail:

A bit of googling suggests that the above statement is wrong: http://groups.google.com/group/android-ndk/browse_thread/thread/2779732b85fef9b3#f0cb99298fa286e1

Some help on debugging Android shared library loading problems: http://mpigulski.blogspot.com/2010/09/debugging-dlopen-unsatisfiedlinkerror.html

And keep in mind this problem when testing:
http://code.google.com/p/android/issues/detail?id=22143

Jonas

---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program.

Reply by email

Felipe Monteiro de Carvalho
to Jonas Maebe
Jonas Maebe wrote
> In Android a dlopen operation without a full path will fail: A bit of googling suggests that the above statement is wrong: http://groups.google.com/group/android-ndk/browse_thread/thread/2779732b85fef9b3#f0cb99298fa286e1

ops, it seams you are correct. In a new test sending only sqlite.so worked properly ... no idea why the first test failed. Probably I was using an old APK or library which tested against libsqlite3.so which fails.

Continuing I have an even much wierder issue. The following code:

procedure TForm3.Button2Click(Sender : TObject) ;
begin
SQLite3Connection1.ExecuteDirect('CREATE TABLE IF NOT EXISTS PRODUTOS ( '+ 'CODPRODUTO INTEGER NOT NULL, '+
'DESCRICAO VARCHAR(50), '+
'PRECO NUMERIC(12,3), '+
'ALTERADO TIMESTAMP, '+

     ' CONSTRAINT PK_PRODUTOS PRIMARY KEY (CODPRODUTO)); ');

end;

Causes a very wierd crash without stacktrace and no way to work around, but only in LCL-CustomDrawn it works fine in LCL-Gtk2:

(gdb) break fpc_raiseexception
Breakpoint 1 at 0x8058d46
(gdb) cont
Continuing.
MouseMove x=47 y=93
MouseMove x=-9 y=63
MouseMove x=-79 y=47
MouseMove x=-57 y=58
MouseMove x=-26 y=63
[Thread debugging using libthread_db enabled]
Cannot find new threads: generic error
Missing debug package(s), you should install: sqlite3-debug-3.7.9-1.1.mga1.i586 (gdb) bt
Target is executing.

This package does not exist. And if I add "cthreads" as the first unit in the project automagically it starts working o.O It looks really wierd that sqlite is using threading behind my back, or is it sqldb?

Very wierd ... the nasty thing here is that cthreads does not work in Android so I will have to first get it into working to have sqlite support. Really unexpected to need threading for sqlite...

Felipe Monteiro de Carvalho

Reply by email

Unknown author
to Felipe Monteiro de Carvalho
Felipe Monteiro de Carvalho wrote 43 lines
On Tue, Apr 3, 2012 at 11:47 AM, Jonas Maebe wrote: >> In Android a dlopen operation without a full path will fail: > A bit of googling suggests that the above statement is wrong: > http://groups.google.com/group/android-ndk/browse_thread/thread/2779732b85fef9b3#f0cb99298fa286e1 ops, it seams you are correct. In a new test sending only sqlite.so worked properly ... no idea why the first test failed. Probably I was using an old APK or library which tested against libsqlite3.so which fails. Continuing I have an even much wierder issue. The following code: procedure TForm3.Button2Click(Sender : TObject) ; begin SQLite3Connection1.ExecuteDirect('CREATE TABLE IF NOT EXISTS PRODUTOS ( '+ 'CODPRODUTO INTEGER NOT NULL, '+ 'DESCRICAO VARCHAR(50), '+ 'PRECO NUMERIC(12,3), '+ 'ALTERADO TIMESTAMP, '+ ' CONSTRAINT PK_PRODUTOS PRIMARY KEY (CODPRODUTO)); '); end; Causes a very wierd crash without stacktrace and no way to work around, but only in LCL-CustomDrawn it works fine in LCL-Gtk2: (gdb) break fpc_raiseexception Breakpoint 1 at 0x8058d46 (gdb) cont Continuing. MouseMove x=47 y=93 MouseMove x=-9 y=63 MouseMove x=-79 y=47 MouseMove x=-57 y=58 MouseMove x=-26 y=63 [Thread debugging using libthread_db enabled] Cannot find new threads: generic error Missing debug package(s), you should install: sqlite3-debug-3.7.9-1.1.mga1.i586 (gdb) bt Target is executing. This package does not exist. And if I add "cthreads" as the first unit in the project automagically it starts working o.O It looks really wierd that sqlite is using threading behind my back, or is it sqldb?

SQLDb does not use threads by itself.

It can be that the sqlite library uses threads, and if a SQLite callback is done from a different thread, then the RTL may present you with a nice error.

Michael.

Reply by email

Felipe Monteiro de Carvalho
to Unknown author

Umm, I merged the pthreads Android fixes from bug 18833 to my built, rebuilt it and added cthreads to my uses clause but I still get the exact same crash inside libsqlite.so =(

Any ideas?

My crash is:

UNCHER] flg=0x10200000 cmp=com.pascal.lcltest/.LCLActivity bnds=[120,148][180,211] }
D/AK8973 ( 78): Compass Start
D/Sensors ( 99): open_akm, fd=111
I/DEBUG ( 70): *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
I/DEBUG ( 70): Build fingerprint:
'htc_wwe/htc_buzz/buzz/buzz:2.2.1/FRG83D/295397:user/release-keys' I/DEBUG ( 70): pid: 2459, tid: 2459 >>> com.pascal.lcltest <<< I/DEBUG ( 70): thread: .pascal.lcltest
I/DEBUG ( 70): signal 4 (SIGILL), fault addr a8b41314 I/DEBUG ( 70): r0 42a01538 r1 42a7b460 r2 81594f78 r3 81580114 I/DEBUG ( 70): r4 42a7b300 r5 a8b41311 r6 813cc8a8 r7 00000001 I/DEBUG ( 70): r8 81345928 r9 814d72f4 10 00000002 fp 00000000 I/DEBUG ( 70): ip bec26d60 sp bec27198 lr 813b0c54 pc a8b41314 cpsr 20000010
I/DEBUG ( 70): #00 pc 00041314 /system/lib/libsqlite.so I/DEBUG ( 70): #01 pc 0001057c /system/lib/libc.so I/DEBUG ( 70):
I/DEBUG ( 70): code around pc:
I/DEBUG ( 70): a8b412f4 a029a697 ffffb60e ffffb613 ffff9d18 I/DEBUG ( 70): a8b41304 4b771290 f7ffb510 bd10fe71 2206b510 I/DEBUG ( 70): a8b41314 f7ff2300 bd10fe6b 4c23b5f0 2300b085 I/DEBUG ( 70): a8b41324 600b9003 1c0d447c fcfef7ce d1381e06 I/DEBUG ( 70): a8b41334 f7c92000 491dfeb1 18601c07 78043064 I/DEBUG ( 70):
I/DEBUG ( 70): code around lr:
I/DEBUG ( 70): 813b0c34 e51b002c e3500000 059f2048 01a00002 I/DEBUG ( 70): 813b0c44 e59f2044 e5925000 e1a0e00f e1a0f005 I/DEBUG ( 70): 813b0c54 e1a01000 e1a00004 eb000022 ebf54fe4 I/DEBUG ( 70): 813b0c64 e24b002c ebf528a3 e3a00000 e50b002c I/DEBUG ( 70): 813b0c74 e51b0064 e3500000 1bf55059 e91ba830 I/DEBUG ( 70):
I/DEBUG ( 70): stack:
I/DEBUG ( 70): bec27158 ab2494dc /system/lib/libskia.so I/DEBUG ( 70): bec2715c bec27488 [stack] I/DEBUG ( 70): bec27160 00000001
I/DEBUG ( 70): bec27164 bec27488 [stack] I/DEBUG ( 70): bec27168 00000001
I/DEBUG ( 70): bec2716c 422c0000 /system/framework/framework.odex I/DEBUG ( 70): bec27170 42180000 /system/framework/framework.odex I/DEBUG ( 70): bec27174 00000085
I/DEBUG ( 70): bec27178 00000000
I/DEBUG ( 70): bec2717c 00004051
I/DEBUG ( 70): bec27180 0000002a
I/DEBUG ( 70): bec27184 42a7b300
I/DEBUG ( 70): bec27188 813b0ba8
/data/data/com.pascal.lcltest/lib/liblclapp.so
I/DEBUG ( 70): bec2718c 813cc8a8
/data/data/com.pascal.lcltest/lib/liblclapp.so
I/DEBUG ( 70): bec27190 42a7b300
I/DEBUG ( 70): bec27194 813b0c20
/data/data/com.pascal.lcltest/lib/liblclapp.so
I/DEBUG ( 70): #00 bec27198 ab249d8c /system/lib/libskia.so I/DEBUG ( 70): bec2719c afd10580 /system/lib/libc.so I/DEBUG ( 70): #01 bec271a0 00146b00 [heap]
I/DEBUG ( 70): bec271a4 ab249d8c /system/lib/libskia.so I/DEBUG ( 70): bec271a8 00000000
I/DEBUG ( 70): bec271ac afd10460 /system/lib/libc.so I/DEBUG ( 70): bec271b0 00000000
I/DEBUG ( 70): bec271b4 42a7b300
I/DEBUG ( 70): bec271b8 813b0ba8
/data/data/com.pascal.lcltest/lib/liblclapp.so
I/DEBUG ( 70): bec271bc 813cc8a8
/data/data/com.pascal.lcltest/lib/liblclapp.so
I/DEBUG ( 70): bec271c0 00000001
I/DEBUG ( 70): bec271c4 81345928
/data/data/com.pascal.lcltest/lib/liblclapp.so
I/DEBUG ( 70): bec271c8 814d72f4
/data/data/com.pascal.lcltest/lib/liblclapp.so
I/DEBUG ( 70): bec271cc 00000002
I/DEBUG ( 70): bec271d0 bec27214 [stack] I/DEBUG ( 70): bec271d4 bec27198 [stack] I/DEBUG ( 70): bec271d8 813b0bd8
/data/data/com.pascal.lcltest/lib/liblclapp.so
I/DEBUG ( 70): bec271dc bec271b4 [stack] I/DEBUG ( 70): bec271e0 bec272c0 [stack] I/DEBUG ( 70): bec271e4 00000001

So analyzing this I see:

A I/DEBUG ( 70): bec27188 813b0ba8
/data/data/com.pascal.lcltest/lib/liblclapp.so
B I/DEBUG ( 70): bec2718c 813cc8a8
/data/data/com.pascal.lcltest/lib/liblclapp.so
C I/DEBUG ( 70): bec27194 813b0c20
/data/data/com.pascal.lcltest/lib/liblclapp.so
A I/DEBUG ( 70): bec271b8 813b0ba8
/data/data/com.pascal.lcltest/lib/liblclapp.so
B I/DEBUG ( 70): bec271bc 813cc8a8
/data/data/com.pascal.lcltest/lib/liblclapp.so
D I/DEBUG ( 70): bec271c4 81345928
/data/data/com.pascal.lcltest/lib/liblclapp.so
E I/DEBUG ( 70): bec271c8 814d72f4
/data/data/com.pascal.lcltest/lib/liblclapp.so
F I/DEBUG ( 70): bec271d8 813b0bd8
/data/data/com.pascal.lcltest/lib/liblclapp.so

Here the 2 top numbers of the address 81xxxxxx are irrelevant

Element A:

003b0b6c <SQLITE3CONN_TSQLITE3CONNECTION_$__ROLLBACKRETAINING$TSQLHANDLE>: .......
3b0b94: e59f1008 ldr r1, [pc, #8] ; 3b0ba4
<SQLITE3CONN_TSQLITE3CONNECTION_$__ROLLBACKRETAINING$TSQLHANDLE+0x38> 3b0b98: eb00007a bl 3b0d88
<SQLITE3CONN_TSQLITE3CONNECTION_$__EXECSQL$ANSISTRING> 3b0b9c: e28dd004 add sp, sp, #4
3b0ba0: e8bd8030 pop {r4, r5, pc}
3b0ba4: 0052638c .word 0x0052638c
003b0ba8 <SQLITE3CONN_TSQLITE3CONNECTION_$__DOINTERNALCONNECT>: 3b0ba8: e1a0c00d mov ip, sp <== JUMP

Element B

003cc8a8 <DB_TDATABASE_$__GETCONNECTED$$BOOLEAN>:
3cc8a8: e5d00059 ldrb r0, [r0, #89] ; 0x59
3cc8ac: e1a0f00e mov pc, lr

Element C

003b0ba8 <SQLITE3CONN_TSQLITE3CONNECTION_$__DOINTERNALCONNECT>: ...........
3b0c0c: eb00d0a6 bl 3e4eac <SQLITE3DYN_INITIALIZESQLITE$ANSISTRING$$LONGINT> 3b0c10: e594005c ldr r0, [r4, #92] ; 0x5c
3b0c14: ebf528d3 bl faf68 <FPC_ANSISTR_INCR_REF>
3b0c18: e24b002c sub r0, fp, #44 ; 0x2c
3b0c1c: ebf528b6 bl faefc <FPC_ANSISTR_DECR_REF>
3b0c20: e284005c add r0, r4, #92 ; 0x5c

Element D

00345928 <STDCTRLS_TBUTTONCONTROL_$__WMDEFAULTCLICKED$TLMESSAGE>:

procedure TButtonControl.WMDefaultClicked(var Message: TLMessage); begin
345928: e1a0c00d mov ip, sp
34592c: e92dd810 push {r4, fp, ip, lr, pc}
345930: e24cb004 sub fp, ip, #4
345934: e24dd034 sub sp, sp, #52 ; 0x34
345938: e50b0030 str r0, [fp, #-48] ; 0x30
34593c: e50b102c str r1, [fp, #-44] ; 0x2c

  Click;
  345940:	e51b0030 	ldr	r0, [fp, #-48]	; 0x30
  345944:	e51b1030 	ldr	r1, [fp, #-48]	; 0x30
  345948:	e5911000 	ldr	r1, [r1]
  34594c:	e59141c0 	ldr	r4, [r1, #448]	; 0x1c0
  345950:	e1a0e00f 	mov	lr, pc
  345954:	e1a0f004 	mov	pc, r4

end;

Element E seams to be data and not code

Element F

003b0ba8 <SQLITE3CONN_TSQLITE3CONNECTION_$__DOINTERNALCONNECT>: 3b0ba8: e1a0c00d mov ip, sp
3b0bac: e92dd830 push {r4, r5, fp, ip, lr, pc}
3b0bb0: e24cb004 sub fp, ip, #4
3b0bb4: e24dd068 sub sp, sp, #104 ; 0x68
3b0bb8: e1a04000 mov r4, r0
3b0bbc: e3a00000 mov r0, #0
3b0bc0: e50b002c str r0, [fp, #-44] ; 0x2c
3b0bc4: e24b2038 sub r2, fp, #56 ; 0x38
3b0bc8: e24b1060 sub r1, fp, #96 ; 0x60
3b0bcc: e3a00001 mov r0, #1
3b0bd0: ebf54efc bl 1047c8 <FPC_PUSHEXCEPTADDR>
3b0bd4: ebf5627b bl 1095c8 <FPC_SETJMP>
3b0bd8: e50b0064 str r0, [fp, #-100] ; 0x64

So shorter we have:

A SQLITE3CONN_TSQLITE3CONNECTION_$__ROLLBACKRETAINING$TSQLHANDLE B DB_TDATABASE_$__GETCONNECTED$$BOOLEAN
C SQLITE3CONN_TSQLITE3CONNECTION_$__DOINTERNALCONNECT A
B
D STDCTRLS_TBUTTONCONTROL_$__WMDEFAULTCLICKED$TLMESSAGE

The assembler is obtained via:

android]$
~/Programas/android-ndk-r7/toolchains/arm-linux-androideabi-4.4.3/prebuilt/linux-x86/bin/arm-linux-androideabi-objdump -S libs/armeabi/liblclapp.so > lclappsym2.txt

Felipe Monteiro de Carvalho

Felipe Monteiro de Carvalho

Reply by email