profile
viewpoint
Eric Sink ericsink SourceGear Illinois, Colorado https://ericsink.com/

ericsink/SQLitePCL.raw 299

A Portable Class Library (PCL) for low-level (raw) access to SQLite

ericsink/wasm2cil 193

A "compiler" that can take a WebAssembly/WASI module and convert it to a .NET assembly

ericsink/LSM 50

Nothing to see here. Move along...

ericsink/DrawnDataGridFramework 14

A framework for drawn grids.

ericsink/blog 1

Content and scripts for my blog

ericsink/ILSupport 1

A Visual Studio extension that provides syntax highlighting for the IL (Intermediate Language) and project templates for C#, F# and Visual Basic that support embedding and calling IL code.

ericsink/Akavache 0

An asynchronous, persistent key-value store

ericsink/dotnet-webassembly 0

Create, read, modify, write and execute WebAssembly (WASM) files from .NET-based applications.

issue commentmicrosoft/LLVMSharp

NuGet package 10.0.0-beta

The raw bindings are auto-generated and since they are generated as 1-to-1 they should always be "correct" and therefore are basically stable from the moment they are checked in (minus minor bugs or issues that can easily be patched).

The higher level bindings however, are a manually written API that has a higher chance of introducing bugs or mistakes and which might not be in the right shape based on user feedback. Breaks introduced here might also be more painful to respond to.

For this reason, I think of these as two rather different pieces of software, and having them be separate packages would make sense to me.

NickStrupat

comment created time in 6 days

issue commentmicrosoft/LLVMSharp

NuGet package 10.0.0-beta

Possibly clueless question:

Is it true that the reason these two things have different notions of stability is because the raw bindings are auto-generated while the proper managed wrapper requires stuff to be done by hand?

NickStrupat

comment created time in 6 days

startedlijiansong/clang-llvm-tutorial

started time in 6 days

issue commentpraeclarum/sqlite-net

AV in sqlite3_finalize called from sqlite3_stmt finalizer

Let me know how this goes.

JoshuaBStevens

comment created time in 15 days

issue commentpraeclarum/sqlite-net

SQLite net pcl throw exception on windows 32 bit (Windows 7 professional)

I don't know if the e_sqlite3 builds are compatible with Windows 7. It is quite possible they are not. Windows 7 is no longer supported by Microsoft as of January 2020.

chuki2

comment created time in 15 days

issue commentpraeclarum/sqlite-net

AV in sqlite3_finalize called from sqlite3_stmt finalizer

Yes, at first glance, the problem you describe with the finalizer might be happening.

"The only disposable sqlite-net object I can see that we're creating is a SQLiteConnection, and we do dispose that (once)."

Apologies in advance for my lack of knowledge on sqlite-net's API.

You're probably using sqlite3_stmt indirectly. Are you using Sqlite3Statement? How are you executing SQL statements or queries? Whatever you're doing, that technique is using a sqlite3_stmt, and something needs to "dispose" it, where in this case, "dispose" actually means the sqlite3_finalize() API. And explicitly doing so should avoid any problem with the finalizer.

JoshuaBStevens

comment created time in 15 days

issue commentpraeclarum/sqlite-net

AV in sqlite3_finalize called from sqlite3_stmt finalizer

Well that's basically the free/dispose/destructor/whatever code for a sqlite3_stmt. SQLite has rather strict rules about such things. I'm guessing perhaps something is getting disposed twice. Or possibly, since this is the finalizer running, maybe the stmt is not getting disposed explicitly, and doing so would avoid the problem.

I just dug up the code for SQLitePCLRaw 1.x and looked it over from the perspective that your stack trace has "ThreadStart" in it. It's possible there might be a race condition in there, but I can't be sure, and 1.1.14 was considered rather stable back in its day.

Note, FWIW, that SQLite uses the word "finalize" in a different sense than the notion of a C# finalizer, and both notions are in play here. You are in a C# finalizer for the sqlite3_stmt wrapper object, and that finalizer is calling sqlite3_finalize(), which is basically the specific free function for a sqlite3_stmt. SQLite requires every stmt to be finalized exactly one time, and in fact, attempts to close the sqlite3 db handle will fail if any unfinalized stmts remain.

Note also that this aspect of the code was completely redone for the 2.0 release. All objects that wrap native pointers are now derived from SafeHandle. But like I said, the transition from 1.x to 2.x is not seamless.

Can you give me further information? What is the app involved here? Is it still true that this problem is being seen only by a customer and you have not reproduced it in house?

JoshuaBStevens

comment created time in 16 days

pull request commentericsink/SQLitePCL.raw

Use FluentAssertions instead of xUnit assertions

I'm rather unsure about this one. I mean, yes, I like FluentAssertions as a concept, but adding a dependency always makes me worry about future hassles.

0xced

comment created time in 21 days

push eventericsink/SQLitePCL.raw

Cédric Luthi

commit sha 19b14bf61bcda836b02fe64c75034b25d1e36069

Improve exception message in fake_xunit Assert.Single

view details

Eric Sink

commit sha e9793ec97a82f41e719ad22a052ecbaab8b58216

Merge pull request #368 from 0xced/fake_xunit-improve-exception-message Improve exception message in fake_xunit Assert.Single

view details

push time in 21 days

push eventericsink/SQLitePCL.raw

Cédric Luthi

commit sha 038b3bb7df1252e04924c8a851fd2359c78c67ba

Fix warnings in Program.fs + exec.fs

view details

Eric Sink

commit sha b906c975d339ae80e09ff3bd99ff4dd747ab4944

Merge pull request #367 from 0xced/fix-warnings Fix warnings in Program.fs + exec.fs

view details

push time in 21 days

push eventericsink/SQLitePCL.raw

Cédric Luthi

commit sha a2dc2dceda351f3c39e45a1125ac933f8e4776ea

Restore version with pre-{timestamp} format Now that version 2.0.4 is released, use the pre format with a timestamp to avoid overwriting the real 2.0.4 NuGet packages in the global packages (i.e. %userprofile%\.nuget\packages or ~/.nuget/packages).

view details

Eric Sink

commit sha 537db9f0a68e3a5baba52b33e17b24a68e8df2f7

Merge pull request #369 from 0xced/pre-timestamp Restore version with pre-{timestamp} format

view details

push time in 21 days

PR merged ericsink/SQLitePCL.raw

Restore version with pre-{timestamp} format

Now that version 2.0.4 is released, use the pre format with a timestamp to avoid overwriting the real 2.0.4 NuGet packages in the global packages (i.e. %userprofile%.nuget\packages or ~/.nuget/packages).

+1 -1

0 comment

1 changed file

0xced

pr closed time in 21 days

issue commentericsink/SQLitePCL.raw

Bring back the SQLitePCLRaw.provider.sqlite3 packages

AFAICT, the packages are both there:

https://www.nuget.org/packages/SQLitePCLRaw.bundle_sqlite3

https://www.nuget.org/packages/SQLitePCLRaw.provider.sqlite3

bricelam

comment created time in 21 days

issue commentericsink/SQLitePCL.raw

System.IO.FileLoadException on windows 32 bit (Windows 7 professional)

Not sure if this is the cause of the problem, but one issue here is that you should not have both SQLite and sqlcipher at the same time. Those are alternatives were you typically choose one or the other, but never both.

chuki2

comment created time in 23 days

created tagericsink/SQLitePCL.raw

tagv2.0.4

A Portable Class Library (PCL) for low-level (raw) access to SQLite

created time in 25 days

release ericsink/SQLitePCL.raw

v2.0.4

released time in 25 days

issue commentericsink/SQLitePCL.raw

2.0.4

2.0.4 has been pushed up to nuget.org.

ericsink

comment created time in a month

push eventericsink/SQLitePCL.raw

Eric Sink

commit sha c6ced2fa3d9cb91a8eef9799dba7064da86f22fe

experimental work on provider.cil

view details

Eric Sink

commit sha 7c6e134d20aaaccaa7773f2debf5781aa6eb4353

rm armeabi e_sqlite3 build from lib.e_sqlite3.android package #356

view details

Eric Sink

commit sha 5115b9795266bb275011a7aaf4c1f28eced4d903

chg build script to look for windows e_sqlite3 builds in v142, while continuing to look for windows e_sqlcipher builds in v141. #356

view details

Eric Sink

commit sha 82fc88f0d12f7e1f4530ec74ee26c5dc88388757

change target netcoreapp3.0 to netcoreapp3.1. #356

view details

Eric Sink

commit sha c9827a0ef71e4a227ca82dfb09c1183f6ee5bca9

add bundle_sqlite3. not tested yet. currently always does DllImport(sqlite3) and only targets netstandard2.0. #356 and #325

view details

Eric Sink

commit sha 005408b22ea700357062e8b6bd67caea0384a108

test for bundle_sqlite3, currently only with fake_xunit and only for netcoreapp3.1. #356 and #325

view details

Eric Sink

commit sha f8d01bfdc843f1b601d02258c91001321e16ac7c

chg all the tests to run with netcoreapp3.1 instead of netcoreapp3.0

view details

Eric Sink

commit sha 88b77ad0b4e99e5be440e2f92a73749cb93bffa6

chg bundle_sqlite3 test to also target net461 and netcoreapp2.1, and add that test to the regular build. #325 and #356

view details

Eric Sink

commit sha e6876a673c3fc025807fbbef7d4eb3f89f529236

build for 2.0.4-pre #356

view details

Eric Sink

commit sha 4e2764f0ddf9b695741c246ff55276ac605b39bd

revert the commit that experimented with cil stuff

view details

Eric Sink

commit sha 628d6cddb785593051b19d28e1c4038aa096516b

rebuild 2.0.4-pre after branch work

view details

Eric Sink

commit sha ae6640978d16de246ec8f8368653d8e0ef450d61

2.0.4

view details

Eric Sink

commit sha 5f7568faebee087cfba41c81122bba2b2e5fb83b

2.0.4 build

view details

Eric Sink

commit sha 54eaed3b67a8b1e473c5b9537d1f73be43086025

Merge branch 'master' of https://github.com/ericsink/SQLitePCL.raw

view details

push time in a month

pull request commentericsink/SQLitePCL.raw

Fix test_other_col_types when running in time zones with a positive offset from UTC

I understand. I was just clarifying.

0xced

comment created time in a month

pull request commentericsink/SQLitePCL.raw

Fix test_other_col_types when running in time zones with a positive offset from UTC

"even if ugly is not meant for production"

FWIW, SQLitePCLRaw.ugly is used in production. My coworkers and I (at SourceGear) use it in a variety of projects.

0xced

comment created time in a month

pull request commentericsink/SQLitePCL.raw

Fix test_other_col_types when running in time zones with a positive offset from UTC

The breaking change to ugly.cs is problematic.

The intent of that code is to convert the given DateTime to a "unix timestamp", which is defined in terms of UTC.

0xced

comment created time in a month

pull request commentericsink/SQLitePCL.raw

Improve build instructions in the README

Sorry for the delay. Thanks.

0xced

comment created time in a month

push eventericsink/SQLitePCL.raw

Cédric Luthi

commit sha 81387f46b0c706e6bc72971a9a1213ffb4edea73

Improve build instructions in the README

view details

Eric Sink

commit sha e2f98778c5a21e6e1081887a75c477317397f27f

Merge pull request #300 from 0xced/improve-build-instructions Improve build instructions in the README

view details

push time in a month

issue commentpraeclarum/sqlite-net

Issue: 'SQL logic error' when upgrading to version 1.7.335 (UWP)

https://github.com/ericsink/SQLitePCL.raw/wiki/UWP-Init

rneeft

comment created time in a month

GollumEvent

issue commentpraeclarum/sqlite-net

Query silently fails (returns 0 results) only on UWP platform after database gets to certain size.

"is there a reasonable way to change the failure behavior in this scenario?"

Maybe, but I'm not sure, and there would be risks.

I too think the behavior is strange, but it arises from deep within SQLite itself.

Another option would be to include these win32_set_directory calls in sqlite-net. Microsoft.Data.Sqlite does this.

For completeness, note that SQLitePCLRaw 1.x did these calls, but they were removing in the 2.x cycle.

tomreich

comment created time in a month

issue commentpraeclarum/sqlite-net

Query silently fails (returns 0 results) only on UWP platform after database gets to certain size.

The repro app doesn't seem to work for me. When I click Start, I don't get that stream of output you posted.

Nonetheless, my guess here is that the UWP build is having trouble because you haven't told SQLite where the sandbox directories are.

Try adding something like this in your app initialization:

            SQLitePCL.Batteries.Init();
            var rc = SQLitePCL.raw.sqlite3_win32_set_directory(
                1, // data
                Windows.Storage.ApplicationData.Current.LocalFolder.Path
                );
            // TODO check rc for errors

            rc = SQLitePCL.raw.sqlite3_win32_set_directory(
                2, // temp
                Windows.Storage.ApplicationData.Current.TemporaryFolder.Path
                );
            // TODO check rc for errors
tomreich

comment created time in a month

issue commentericsink/SQLitePCL.raw

Migrate packages.config -> PackageReference causes 'VTable setup of type SQLitePCL.SQLite3Provider_sqlite3 failed'

There are multiple problems here, and I'm not sure which of them might be causing the symptom you are seeing.

(1) If you explicitly include SQLitePCLRaw.bundle_sqlcipher, you do need to explicitly include SQLitePCLRaw.lib.sqlcipher.ios_unified.static or SQLitePCLRaw.provider.internal.ios_unified. They will come in as transitive dependencies, and doing it that way will keep them matched to the version SQLitePCLRaw.bundle_sqlcipher.

(2) If you want to use SQLitePCLRaw 1.x (which is no longer actively maintained) instead of 2.x (which contains breaking changes),then I suggest you use version 1.1.14.

(3) You have bundle_sqlcipher as well as bundle_green, but you should never have more than one SQLitePCLRaw bundle package. It is a surprising that this ever worked. It looks like what you want is bundle_sqlcipher, and that bundle_green is coming in transitively through sqlite-net-pcl which is coming in transitively through SQLiteNetExtensions. The usual advice is to use sqlite-net-base (1.6.x, in your case) instead of sqlite-net-pcl. But since bundle_green is coming in transitively, you probably need to do something like this to get rid of it:

<PackageReference Include="SQLitePCLRaw.bundle_green" Version="1.1.14" ExcludeAssets="All" />
zoli13

comment created time in a month

issue commentericsink/SQLitePCL.raw

How to support libe_sqlite3.so for mips64 ARCH ?

@sunny868 Please verify that the linux-mips64 builds for e_sqlite3 and e_sqlcipher in 2.0.4-pre20200828111558 are working properly. These are not the builds you committed, but rather, were re-done by my build system.

sunny868

comment created time in a month

issue commentericsink/SQLitePCL.raw

2.0.4

More detailed draft of release notes for 2.0.4:

  1. The e_sqlite3 builds are now at SQLite 3.33.0 #350
  2. The e_sqlcipher builds remain unchanged
  3. Windows e_sqlite3 builds are now done with tools v142
  4. Android e_sqlite3 builds are now done with NDK r21d, and armeabi is no longer supported.
  5. Builds of e_sqlite3 and e_sqlcipher are now included for linux-mips64 (thanks to @sunny868, see #360)
  6. Target netcoreapp3.1 instead of netcoreapp3.0
  7. New bundle_sqlite3 package #325
  8. Fix #321 crash bug

Notable things NOT in 2.0.4:

  • No changes related to the use of NativeLibrary.Load and problems with AssemblyLoadContext #343
  • No changes related to sqlite3_isexplain and the corresponding problems with UWP builds #346
  • No support for recent SQLite features like uint, decimal, ieee754, because they are not in the SQLite amalgamation
  • No fix for Xamarin.Mac problem finding the native code library #358
  • Still targeting net461 even thought there are cases where saying net472 would apparently be more correct #349
ericsink

comment created time in a month

issue commentericsink/SQLitePCL.raw

System.AccessViolationException at System.Text.UTF8Encoding.GetCharCount(Byte*, Int32, System.Text.DecoderNLS)

The 2.0.4-pre release I uploaded to nuget.org today contains the fix for this problem.

jay1891

comment created time in a month

pull request commentdotnet/efcore

Update to SQLitePCLRaw 2.0.4-pre20200828111558

I'll decline to give any advice on your process. :-) But for additional background:

The main purpose of this 2.0.4-pre is to let @bricelam verify that certain changes in my 2.0.4 work are working as expected. If he gives the thumbs-up on this -pre, I expect to do 2.0.4 final next week. If he finds a problem, I still expect to be able to fix it and do 2.0.4 final next week. I suspect that this PR is simply Brice's most convenient way to do the verification I just mentioned.

bricelam

comment created time in a month

pull request commentdotnet/efcore

Update to SQLitePCLRaw 2.0.4-pre20200828111558

"How confident are we"

Very confident.

bricelam

comment created time in a month

issue commentericsink/SQLitePCL.raw

2.0.4

2.0.4-pre20200828111558 has just been pushed up to nuget.org.

Incomplete and off-the-cuff release notes:

  1. SQLite 3.33.0
  2. bundle_sqlite3
  3. All netcoreapp3.0 TFMs updated to netcoreapp3.1
  4. e_sqlite3 and e_sqlcipher builds for linux-mips64 (thanks to @sunny868, see #360)
ericsink

comment created time in a month

push eventericsink/SQLitePCL.raw

Eric Sink

commit sha e6876a673c3fc025807fbbef7d4eb3f89f529236

build for 2.0.4-pre #356

view details

push time in a month

push eventericsink/cb

Eric Sink

commit sha 6d5e562c812983d7004d01800d890f2be73ccfe4

rebuild linux cross

view details

push time in a month

push eventericsink/cb

Eric Sink

commit sha df4878b1bdef51c3910aadd9f07443005ff982d7

rebuild linux x64 and x86

view details

push time in a month

issue commentericsink/SQLitePCL.raw

How to support libe_sqlite3.so for mips64 ARCH ?

Yes, I’m planning to do a 2.0.4-pre release soon, which should include the mips64 builds.

sunny868

comment created time in a month

issue commentericsink/SQLitePCL.raw

How to support libe_sqlite3.so for mips64 ARCH ?

What is the RID for linux mips64 for .NET Core?

sunny868

comment created time in a month

pull request commentericsink/cb

Support mips64 architecture on Linux platform

Your build script changes and the cross-compiler package you mentioned seem to be working fine for me. Thanks!

sunny868

comment created time in a month

push eventericsink/cb

Eric Sink

commit sha 94f182bf8a2c0121f6e3d2ce57801e95ca075011

add mips64 toolchain to setup_linux.sh

view details

push time in a month

push eventericsink/cb

Sunguoyun

commit sha ad823052ae0a4106d516d38dae416c07689d646e

Support mips64 architecture on Linux platform

view details

Eric Sink

commit sha 56325a5fbefa6fc9dbf8bc505c2137415cf87dcf

Merge pull request #1 from sunny868/mips64 Support mips64 architecture on Linux platform

view details

push time in a month

PR merged ericsink/cb

Support mips64 architecture on Linux platform
+10 -1

3 comments

3 changed files

sunny868

pr closed time in a month

push eventericsink/cb

Eric Sink

commit sha 47d5bd76b204bda2bab9007d45aa59e0471aa246

setup_linux.sh

view details

push time in a month

issue commentericsink/SQLitePCL.raw

build failed on Linux Platform

Ah, I see.

AFAIK, the build doesn't work on anything but Windows. I've tried in the past, and it's close, but not quite there.

Currently, the build must be run on Windows, and in a Developer Command Prompt shell for Visual Studio 2019.

sunny868

comment created time in a month

pull request commentericsink/cb

Support mips64 architecture on Linux platform

So I see you have included the .so files. Does that mean this build worked for you?

What platform did you do the build on? Ubuntu? Which version?

If Ubuntu, which package did you install for the cross compiler?

sunny868

comment created time in a month

issue commentericsink/SQLitePCL.raw

How to support libe_sqlite3.so for mips64 ARCH ?

It is not a simple task at all, but here are a few tips if you really want to try:

The build files are generated by a program, the source code for that program is in cb.cs.

Compiling cb.cs is done by simply running csc or mcs. There is no csproj file.

There are no Makefiles. Each script is either a bash script of .bat file, depending on the platform.

cb.cs has virtually no comments to help. But the basic idea is to modify cb.cs to also generate linux mips64 stuff.

You'll need to figure out which ubuntu packages have the cross-compilers for mips64 and how to invoke them. I have no idea about this.

The source code for sqlite is in the cb/sqlite. It's sqlite3.c.

You could skip the whole cb step by just figuring how to invoke the right cross-compiler and then drop the resulting .so file in the right place.

Then you'll need to modify the nuspec scripts in the SQLitePCL.raw repo to include the new file. And so on.

This will be extremely difficult. These parts of SQLitePCL.raw were never designed to be easy for others to work on. As far as I know, nobody else has ever modified these scripts.

sunny868

comment created time in a month

issue commentericsink/SQLitePCL.raw

How to support libe_sqlite3.so for mips64 ARCH ?

I don't currently build e_sqlite3 for mips64.

The build system for e_sqlite3 is in a separate repo:

https://github.com/ericsink/cb

What platform are you needing? Linux? Android?

sunny868

comment created time in a month

issue commentericsink/SQLitePCL.raw

e_sqlite3 library not found in Xamarin.Mac

"can't imagine I'm the only one using Xamarin.Mac"

Probably not, but it's close. :-) I've been surprised how few people have mentioned this problem.

nvanlaerebeke

comment created time in a month

issue commentericsink/SQLitePCL.raw

e_sqlite3 library not found in Xamarin.Mac

Actually I'm quite sure there IS a workaround, I just don't remember how to do it. :-)

The problem is conceptually rather simple -- we just need the e_sqlite3 library to end up where the Mac code can find it. Once that happens, everywhere works.

I'll take a closer look again and see if I can figure something out.

nvanlaerebeke

comment created time in a month

issue commentericsink/SQLitePCL.raw

e_sqlite3 library not found in Xamarin.Mac

This is a known problem, although, strangely, there appears to be no issue for it.

It was mentioned in the 2.0 release notes:

https://github.com/ericsink/SQLitePCL.raw/blob/master/v2.md

nvanlaerebeke

comment created time in a month

issue commentpraeclarum/sqlite-net

Huge performance loss moving from Cipher 3 to Cipher 4 (Android device)

@sjlombardo Hmmm, that doesn't sound right. I mean, I'm pretty sure I was using libtomcrypt for the previous set of builds as well. So perhaps I shouldn't have mentioned this, since the underlying crypto provider didn't change on the upgrade.

Still, relative to Zetetic official builds, I assume that libtomcrypt might be causing a perf difference, even if that concern is not in play for this issue.

RSF-Deus

comment created time in a month

issue commentpraeclarum/sqlite-net

Huge performance loss moving from Cipher 3 to Cipher 4 (Android device)

Note that my unofficial e_sqlcipher builds use libtomcrypt, which I have assumed could be causing performance differences.

RSF-Deus

comment created time in a month

issue commentpraeclarum/sqlite-net

SQLiteException: file is not a database

@praeclarum if this starts looking a problem down near SQLitePCLRaw, lemme know how I can help.

fgiacomelli

comment created time in a month

issue commentpraeclarum/sqlite-net

sqlite-net-sqlcipher 1.7.335 Android encryption

No worries. Cheers.

jklobut

comment created time in a month

issue commentpraeclarum/sqlite-net

sqlite-net-sqlcipher 1.7.335 Android encryption

"I installed sqlite-net-pcl and sqlite-net-sqlcipher"

You should only install ONE of these at a time.

jklobut

comment created time in a month

issue commentpraeclarum/sqlite-net

sqlite-net-sqlcipher 1.7.335 Android encryption

Suggestion: In the Android app, try running the sqlite statement PRAGMA cipher_version and see what it gives. This should tell you if your Android app is actually talking to sqlcipher, or if something got messed up in the build configuration.

jklobut

comment created time in a month

issue commentpraeclarum/sqlite-net

sqlite-net-sqlcipher 1.7.335 Android encryption

Oops, sorry, I misread.

jklobut

comment created time in a month

issue commentpraeclarum/sqlite-net

sqlite-net-sqlcipher 1.7.335 Android encryption

Current builds of e_sqlcipher are built on SQLCipher 4.x, which by default does not work with SQLCipher 3.x databases.

https://discuss.zetetic.net/t/upgrading-to-sqlcipher-4/3283

jklobut

comment created time in a month

issue commentericsink/SQLitePCL.raw

Bring back the SQLitePCLRaw.provider.sqlite3 packages

OTOH, if this package helped the Unity on iOS case, then it seems that bundle_green would too.

bricelam

comment created time in a month

issue commentericsink/SQLitePCL.raw

Problem with Sqlite-net 1.7.335

Are you suggesting that the space in the path is the problem?

Does your code work if you change the path to something that does not include a space?

Microlife-JasonLi

comment created time in a month

issue commentericsink/SQLitePCL.raw

Bring back the SQLitePCLRaw.provider.sqlite3 packages

@bricelam I've added bundle_sqlite3.

I'm not sure if there are any edge cases that matter to you which I might be forgetting about, but the bundle right now is pretty darn simple. It only targets netstandard2.0, and it always just does DllImport of "sqlite3" (as opposed to NativeLibrary.Load or any other dynamic approach).

It has a test app which runs against netcoreapp2.1, netcoreapp3.1, and net461, and on my machine, it passes all my usual tests against all 3 targets, with the 64 bit sqlite3.dll obtained from sqlite.org. I haven't tried anything like using Linux against whatever sqlite package is part of Ubuntu, for example.

None of this is up on nuget yet, but I'll probably be doing a 2.0.4-pre at some point soon for testing purposes.

Feedback welcome.

bricelam

comment created time in a month

push eventericsink/SQLitePCL.raw

Eric Sink

commit sha c9827a0ef71e4a227ca82dfb09c1183f6ee5bca9

add bundle_sqlite3. not tested yet. currently always does DllImport(sqlite3) and only targets netstandard2.0. #356 and #325

view details

Eric Sink

commit sha 005408b22ea700357062e8b6bd67caea0384a108

test for bundle_sqlite3, currently only with fake_xunit and only for netcoreapp3.1. #356 and #325

view details

Eric Sink

commit sha f8d01bfdc843f1b601d02258c91001321e16ac7c

chg all the tests to run with netcoreapp3.1 instead of netcoreapp3.0

view details

Eric Sink

commit sha 88b77ad0b4e99e5be440e2f92a73749cb93bffa6

chg bundle_sqlite3 test to also target net461 and netcoreapp2.1, and add that test to the regular build. #325 and #356

view details

push time in a month

issue commentericsink/SQLitePCL.raw

2.0.4

Worth noting: for 2.0.4, all nuget packages that targeted netcoreapp3.0 have been changed to netcoreapp3.1.

ericsink

comment created time in a month

push eventericsink/SQLitePCL.raw

Eric Sink

commit sha 82fc88f0d12f7e1f4530ec74ee26c5dc88388757

change target netcoreapp3.0 to netcoreapp3.1. #356

view details

push time in a month

issue commentericsink/SQLitePCL.raw

2.0.4

Things worth noting about the e_sqlite3 builds for 2.0.4:

  1. The build system for e_sqlite3 is actually over in the very-poorly-named sibling repo: https://github.com/ericsink/cb
  2. The corresponding SQLite version will be 3.33.0
  3. For Windows, these builds are compiled with Visual Studio 2019, toolset v142, whereas previously the builds were made with toolset v141.
  4. For Mac and iOS, these builds are compiled with the exact same tooling as previous versions.
  5. For Android, these builds are compiled with NDK r21d, whereas previously the builds were made with NDK r13b. The "armeabi" build has been removed, as it was removed from the NDK back in r17, and SQLite 3.33.0 no longer compiles under NDK r13b.
  6. For Linux, these builds are compiled with the exact same tooling as previous versions, hosted on Ubuntu 16.04. I tried updating to a build host based on 18.04, but the gcc-multilib package (which is the usual way to add support for compiling 32-bit) is incompatible with the cross-compiler packages (which I use for arm, aarch64, and musl), and now does not seem to be the time to sort that out.
  7. The build settings (the -DSQLITE_etc) for these builds are the same as previous builds.
  8. Tentatively, I plan for the e_sqlcipher builds for 2.0.4 to remain exactly the same builds as 2.0.3, which means they remain based on SQLite 3.28.0.
ericsink

comment created time in a month

push eventericsink/SQLitePCL.raw

Eric Sink

commit sha 7c6e134d20aaaccaa7773f2debf5781aa6eb4353

rm armeabi e_sqlite3 build from lib.e_sqlite3.android package #356

view details

Eric Sink

commit sha 5115b9795266bb275011a7aaf4c1f28eced4d903

chg build script to look for windows e_sqlite3 builds in v142, while continuing to look for windows e_sqlcipher builds in v141. #356

view details

push time in a month

push eventericsink/cb

Eric Sink

commit sha 93dd03d0549d0b35c8ab36431c628227db8a28d4

readme for libtomcrypt

view details

Eric Sink

commit sha 44f1ff7de21f79aaaccaa440ea7d05e51021dedc

Merge branch 'master' of https://github.com/ericsink/cb

view details

push time in a month

push eventericsink/cb

Eric Sink

commit sha 93055e90f568733a92bb6797c55918fcff0ed082

remove armeabi from android build for e_sqlite3

view details

Eric Sink

commit sha 9b6012f9c95c2657f2d274427b0fe1a38736b135

tweak README for update to ndk r21d

view details

Eric Sink

commit sha f6cf7d209d4ca558dc5cac7279da014005181327

rebuild e_sqlite3 for android for 3.33.0, now using latest ndk (r21d), and armeabi is gone. SQLite 3.33.0 no longer builds under armeabi due to missing symbols, and armeabi has been deprecated by the NDKs for a long time now. ericsink/SQLitePCL.raw#356

view details

Eric Sink

commit sha d26ea0e5f4d6f587ff48b59ecd4f656571f594b2

Merge branch 'master' of https://github.com/ericsink/cb

view details

push time in a month

push eventericsink/cb

Eric Sink

commit sha d4e1b1bfe466c8aabd5fc598545bdefd96870b11

rebuild ios and mac e_sqlite3 for 3.33.0 ericsink/SQLitePCL.raw#356

view details

push time in a month

push eventericsink/cb

Eric Sink

commit sha a35322e022dbc220af685d37d5d95a8d0fd49433

rebuild linux e_sqlite3 for 3.33.0 ericsink/SQLitePCL.raw#356

view details

push time in a month

push eventericsink/cb

Eric Sink

commit sha 1f4a4fab345b7bcb13cf614d4869cd3eb5eb5b2d

update sqlite amalgamation to 3.33.0 ericsink/SQLitePCL.raw#356

view details

Eric Sink

commit sha bbabdc572b62360223a9f8010d97179f35dc57c1

update windows builds to use Visual Studio 2019

view details

Eric Sink

commit sha 9f803bd7f806527e3bbae5031c9931c52540d0f3

update windows e_sqlite3 builds to SQLite 3.33.0, now compiled with Visual Studio 2019 v142. ericsink/SQLitePCL.raw#356

view details

push time in a month

issue commentericsink/SQLitePCL.raw

2.0.4

The uint, decimal, and ieee754 features look appealing, but it looks like none of them are in the amalgamation yet. I won't rule this out, but supporting another set of native DLLs causes a feeling of dread. :-)

ericsink

comment created time in 2 months

issue commentericsink/SQLitePCL.raw

2.0.4

Should 2.0.4 do something related to #343 ?

For example, we could move a little further away from NativeAssembly.Load() back toward regular DllImport. But this can cause problems for .NET Framework because of the need to "pre-load" the native DLL to deal with things like 32-vs-64 bit issues.

Or we could just ignore the issue and wait for the issue to get resolved in the platform (see the issue linked in #343

ericsink

comment created time in 2 months

issue commentericsink/SQLitePCL.raw

2.0.4

@sjlombardo Yeah so probably 3.33 then...

ericsink

comment created time in 2 months

issue commentericsink/SQLitePCL.raw

2.0.4

Probably should add the bundle described in #325

ericsink

comment created time in 2 months

issue commentericsink/SQLitePCL.raw

2.0.4

For 2.0.4 consideration: Does something need to be done regarding #346 ?

Adding support for stmt_isexplain broke compatibilty with winsqlite3.dll.

Should support for stmt_isexplain be removed?

Should additional code be added to cope with the presence or absence of stmt_isexplain?

Should we do nothing and wait for winsqlite3.dll to catch up.

ericsink

comment created time in 2 months

issue commentericsink/SQLitePCL.raw

2.0.4

@bricelam I wish to make sure that the timing of my 2.0.4 release is causing no trouble with respect to the timing of the .NET 5.0 release cycle. Please feel free to advise.

ericsink

comment created time in 2 months

issue openedericsink/SQLitePCL.raw

2.0.4

This is the place to summarize or talk about what should go in 2.0.4.

The one thing I'm sure is needed is updating the e_sqlite3 builds to 3.32.x.

created time in 2 months

issue commentpraeclarum/sqlite-net

Support for SQLitePCLRaw 2.0 version

When you updated to SQLitePCLRaw 2.0, the version of sqlcipher was updated as well, from 3.x to 4.x. Version 4 does not work with 3.x databases by default. This may be the problem you are seeing.

I cannot provide much support for using SQLCipher. I usually recommend that people purchase SQLCipher builds/support from Zetetic.

KalyanXamarin

comment created time in 2 months

issue commentpraeclarum/sqlite-net

Support for SQLitePCLRaw 2.0 version

1.7 is out of beta, so you can use the released version

KalyanXamarin

comment created time in 2 months

issue commentericsink/SQLitePCL.raw

Cannot encrypt existing and populated database with SQLitePCLRaw.bundle_e_sqlcipher

Maybe I'm reading this wrong, but isn't this how PRAGMA key is supposed to work?

https://www.zetetic.net/sqlcipher/sqlcipher-api/#PRAGMA_key

vmoglan

comment created time in 2 months

issue commentericsink/SQLitePCL.raw

Unity iOS Support

@corytrese That's interesting.

So if you're just adding sqlite3.c to your project directly, then you don't need my e_sqlite3 lib package at all.

So maybe just try this provider:

https://www.nuget.org/packages/SQLitePCLRaw.provider.internal/

With this line to initialize it:

SQLitePCL.raw.SetProvider(new SQLitePCL.SQLite3Provider_internal());
Heged19

comment created time in 2 months

issue commentericsink/SQLitePCL.raw

Unity iOS Support

Mostly I'm hoping you folks will forgive MY ignorance with respect to Unity. :-)

There may still be differences in your snippet that I don't understand. But as far as I can tell, the approach is the same one I'm using.

It seems like @Heged19 was pretty close to getting this to work, except perhaps for the linking problem.

Heged19

comment created time in 2 months

issue commentericsink/SQLitePCL.raw

Unity iOS Support

Yes, I could add a provider specifically for IL2CPP.

But the snippet you posted looks almost exactly like one of the existing providers. It seems unlikely that the difficulties you all are experiencing are at this layer.

For example, the approach in the snippet will have exactly the same kinds of issues with respect to the linker as discussed above. The linker problem, although frustrating, is relatively straightforward compared to the marshaling issues.

That said, the issue of marshaling delegates would be a separate problem, and perhaps one that IL2CPP simply cannot handle.

Heged19

comment created time in 2 months

issue commentericsink/SQLitePCL.raw

Unity iOS Support

Yes, that method is empty.

Heged19

comment created time in 2 months

issue commentericsink/SQLitePCL.raw

Unity iOS Support

My SQLitePCLRaw.lib package for iOS contains a method which is called by the bundle package to prevent the linker from stripping away the assembly. It calls:

SQLitePCL.lib.embedded.Init();
Heged19

comment created time in 2 months

issue commentericsink/SQLitePCL.raw

Investigate: Is there something wrong with our usage of S.R.I.NativeLibrary?

Link to related issue:

dotnet/runtime#13819

ericsink

comment created time in 2 months

issue commentericsink/SQLitePCL.raw

Native assembly loading outside of application (e_sqlite3.dll)

Link to related issue:

dotnet/runtime#13819

yojagad

comment created time in 2 months

issue commentericsink/SQLitePCL.raw

Native assembly loading outside of application (e_sqlite3.dll)

Did you try the FreezeProvider call?

yojagad

comment created time in 2 months

issue commentericsink/SQLitePCL.raw

Native assembly loading outside of application (e_sqlite3.dll)

"Could the bundle there be conflicting with this somehow?"

The workaround we've been discussing requires calling SetProvider(). If a bundle_e_sqlite package is getting into the project where we are trying to use that workaround, then yes, that's a problem.

Another thing you could try is adding a call to FreezeProvider() just below the SetProvider() call:

SQLitePCL.raw.SetProvider(new SQLitePCL.SQLite3Provider_e_sqlite3());
SQLitePCL.raw.FreezeProvider();

If your SetProvider() call happens first, then the FreezeProvider() call should prevent others from messing it up.

yojagad

comment created time in 2 months

issue commentericsink/SQLitePCL.raw

Native assembly loading outside of application (e_sqlite3.dll)

provider.dynamic_cdecl is not supposed to be around anymore. Somewhere it is still sneaking in. Maybe some other dependency. Maybe try a clean and full rebuild.

yojagad

comment created time in 2 months

issue commentericsink/SQLitePCL.raw

Native assembly loading outside of application (e_sqlite3.dll)

Yeah, that's what I'm talking about. You have a reference to Microsoft.Data.Sqlite, which is bringing in (as a dependency) SQLitePCLRaw.bundle_e_sqlite3, and you don't want that.

Try changing it to Microsoft.Data.Sqlite.Core instead.

yojagad

comment created time in 2 months

issue commentericsink/SQLitePCL.raw

Native assembly loading outside of application (e_sqlite3.dll)

You appear to still have a reference somewhere to a bundle package. I'm guessing it's probably coming in through Microsoft.Data.Sqlite?

Try Microsoft.Data.Sqlite.Core instead?

yojagad

comment created time in 2 months

more