Long Time No Post and PetaPoco v5
Monday, 21 May 2012
It's been over six months since my last post here because I've been slack and I've been busy. Also, I've found a little time to spend on PetaPoco.
After a very busy 2011, I took the first couple of months of 2012 off and did pretty much nothing - at least nothing to do with software development - just alot of unwinding and I feel all the better for it.
Since then I've been working on a few different projects, including new version of Cantabile, some contract work doing mobile development for iOS and Android, and more recently I've been playing with OpenGL, Mono, MonoTouch and MonoDroid with thoughts of maybe doing a mobile game (it's been along time since I did any games programming).
The question I keep getting asked however is when will there be a new version of PetaPoco? Well firstly I'm somewhat reluctant to change PetaPoco too much - simply because I want it to stay as simple as possible and well, I like it the way it is. If you were hoping to see PetaPoco grow into something bigger, more feature rich, then I'm sorry to disappoint - but that's simply missing the point of what PetaPoco is. Also since none of the projects I'm currently working on use PetaPoco it's easy to just leave well enough alone.
That said I have found some time to start on a new version - which will be called v5. Even though there's nothing major about it, I have opted for a major version number bump because it won't be drop-in backwardly compatible.
Dropping the Single C# File Approach
The first thing I've done with v5 is to forget about trying to maintain the whole thing in a single C# source file. Similarly I'm dropping the fascination with trying to keep the line count as small as possible. These ideas served well to keep PetaPoco tiny during its inception, but really serve no practical purpose moving forward.
So the project is now a separate assembly and the original PetaPoco.cs source file has been split up into a series of source files (one class/interface/enum per file) and grouped into folders - all of which means the project is much easier to navigate around and therefore easier to maintain.
But I Like a Single C# File
Although for maintenance purposes having the code base in separate source files is a great, from the point of view of using it in a project there's a certain appeal to being able to drop in a single source file and not having to worry about assembly dependencies. So, I've knocked together a tool that munges everything back into a stand-alone PetaPoco.cs file. Best of both worlds!
CSJ (for CSharp Join) is a simple command line tool which you'll find in the PetaPoco solution that can join a bunch of C# files into a single file. It works well enough for what I need for PetaPoco, but might work for other simple projects too.
CSJ is run automatically by the PetaPoco project during it's post-build phase.
XML Documentation
Because of my previous line count obsession I had been reluctant to do XML documentation comments for PetaPoco. Now that it's broken up into separate files I now feel like there's "room" for these.
The XML documentation on all the public parts of PetaPoco is now complete, however... my eyes glazed over and my brain shut-down as I did it - there could be some weird comments in there.
New Mapper API
Turning PetaPoco into an assembly has some ramifications - particularly in relation to the static Database.Mapper property. As an assembly, there's the possibility of it being used by two other assemblies at the same time and if both wanted to register type mappers it would be a case of last in best dressed.
So the old Database.Mapper property has been removed and replaced with a new Mappers class through which IMapper's can be registered - associating a mapper with either a specific POCO type, or for all the types in an assembly.
Aside from how they're registered the IMapper interface has also had a bit of clean up and PetaPoco's automatic built-in mapping is now implemented as a mapper itself. See the IMapper interface for more on this.
ForceToUtc Changes
This feature was always a bit of hack. Some DB providers return date time values with a DateTimeKind of Unknown. The ForceToUtc option simply changes the returned values from Unkown to DateTimeKind.Utc - without actually changing the date value.
In PetaPoco 4, this was an option on the Database class - now it's an attribute of the POCO property. eg: Column[ForceToUtc = true]
Another important change... ForceToUtc used to be true by default, now it's false.
Renaming Things
As mentioned above, some things have been renamed for consistency:
PrimaryKeyAttribute properties:
- Value -> PrimaryKey
- sequenceName -> SequenceName
- autoIncrement -> AutoIncrement
TableNameAttribute properties:
- Value -> TableName
Tuples for Dictionary Keys
In PetaPoco v4 I was lazy with keying the internal dictionaries that PetaPoco uses. Instead of declaring key objects and writing to associated Hash calculations I just dumped all the key values into a string and used that as a key. With later versions of the runtime an easier (and I presume faster) approach is to use Tuples - which is what PetaPoco now does.
The downside to this is that PetaPoco now requires .NET 4.0. To get it running on earlier versions should just require re-implementing some compatible Tuple classes, but I just haven't got around to it yet.
Other Internal Changes
Internally, PetaPoco has also had a bit of rework/refactor/cleanup:
-
The old DBType property has been replaced with an abstract DatabaseType class that provides database engine specific functionality. Old
if this database do this else do thattype code has been replaced with calls to the DatabaseType class. (See the DatabaseTypes folder in the solution for more on these) -
There's a new Cache classe that simplifies the caching of various objects within PetaPoco.
-
Lots of other minor bits and pieces.
Pull Requests Merged
At this point, the main functional changes are the following pull requests that have now been merged:
- Replace column names that conflict with CSharp keywords automatically
- Add Page
override to support user defined SQL for Count and Paging - Fix for issue #91 - Template was selecting the clustered index, not the primary key.
- Check connection state prior to Open() to avoid error: System.InvalidOperation
- Change rxOrderBy Regex and logic to be less greedy during Page
query rewrite to support sub-select ordering - Add IsNew() Support for PK of Type Guid
- Fix: ensure the primary key value is returned from Insert() when the primary key column is not an autoincrement.
- Fix for Issue #81
- Speeded up mapping with types containing enums.
- Added fix for uint columns in SQLite
- Allow Nullable T in ExecuteScalar
- Fix for crash which would occur when trying to save more than 4000 characters into a SQLServerCE NText column
(Tip: if you want to see something fixed in PetaPoco, your best chance is to submit a pull request).
Upgrade CheckList
Here's a quick check list of things to consider/remember if upgrading from PetaPoco 4 to 5.
- If you need pre .NET 4 support, abort now. (or write some tuple classes as described above)
- Get the latest version from GitHub on the v5 (not master) branch
-
Decide on whether you prefer the Assembly or Single Source File version of PetaPoco.
- If you want the assembly version you'll need to build the PetaPoco project in the supplied solution. Remove PetaPoco.cs from your existing project and add the assembly reference.
- If you want to continue using the Single Source File, just copy the new PetaPoco.cs over the old one.
- If you're using custom mappers, you'll need to switch from assiging the Database.Mapper property to calling Mappers.Register().
- You might get build errors on some attribute properties that have been renamed. Typically these are just upper/lower case issues and should be easily fixed.
- If you're using ForceToUtc option, be warned. You'll need to explicitly decorate all DateTime Columns with the appropriate attribute or, write a custom mapper to do it.
- Cross your fingers :)
Availability
All of this is available in a new 'v5' branch on GitHub, but shouldn't be considered production quality. Feel free to grab it and give it a go and let me know what you think, but more importantly what's broken. Once I'm confident that it's stable I'll put up a new build to Nuget.
What's this about an iOS Game?
Yes, I've been doing some iOS game development. Nothing too fancy, but certainly fun - I might post some info about this soon.
25 Comments
Leave a comment
Brad,
Thank you so much for picking up PetaPoco again. We're still using it but had moved over to schotime's branch (https://github.com/schotime/PetaPoco) .
Your new approach looks better in the long run indeed. Thanks for merging the pull requests in. I'll be trying it out as soon as I can.
Thanks Brad, six months isn't too long to wait for an update and it all worked great as you left it anyway. We've just started to use Petapoco at work to replace EF where it's overkill. I'll be looking at the new branch over the next week.
Also looking forward to the iOS game posts as I want to create a game for my next side project
I agree that PetaPoco should remain as simple as possible. We came here for that and PetaPoco currently covers all my needs. That said, some projects, like SQLinq.Dapper (http://sqlinq.codeplex.com/) are interesting and could enrich PetaPoco as well.
Awesome! Love me some PetaPoco. This is a long awaited update and happy that it is its own assembly now.
Brad,
It's great to see you back on the air. PetaPoco has worked like a charm for me in several production systems over the last year. Of course I couldn’t resist tweaking the guts a little, like adding bindings for some of my custom types and modifying the T4 templates to generate separate files for each model and repository class (with a read only version for views). Your refactor into multiple files will make life even easier, but as you said, it was working pretty darn well already!
Thank you again for this outstanding application! It’s a peach!
Hi,
I'm wondering if any of these changes are breaking for Mono usage? We're hoping to switch over to Mono later in the year and would like to keep using PetaPoco.
Hi,
Can merge the multi-result feature from schotime's folk?
I've just started using this PetaPoco, with fantastic results; so far it has to be one of the best microORM implementations I have used. I applaud it's intuitive nature in knowing what I want for results and it’s simple decorative attributes.
However, something horrified me yesterday. EnableAutoSelect, due to what was a bug, I placed in an “exec …” command, by it prepended the command with a select statement without a where clause. This made my website show all records in the table, which is really bad, because the records are shared with customers. Its mistakes like these that have gotten banks and other institutions into a lot of trouble because they accidently selected all results in a table to the page.
I would really like to see this feature removed or at least switched off, it has terrible ramifications should it be improperly used and it seems likely. Not all developers are as safe conscious as others.
Despite this, please keep up the good work, this has much more promise than many other solutions out there.
@James - where you wrote "exec ..." next time write ";exec ..." that will stop the auto select.
Hi Brad, I only have one question.. where is the NuGet package? :) I waited a while before asking in case it was just inprog.
Of course anyone could put up a package but its better if the chief dev does it, in my opinion at least.
I will be creating some "extensions" for my next project and its nice if I can put the dependency into my packages.
Thanks,
Another vote for merging the fork from schotime's branch into V5 I see great extension from that fork and would definitely like to see those included in the new version
Thanks and Great Work!!!
I think the NuGet package should be in binary form. That would make it just as easy for VB users as C#. You can't just go dropping .cs files into VB projects. PetaPoco.dll, on the other hand, would work just fine.
Any updates on when Petapoco v5 will be made a Nuget package?
+1 vote for composite primary key with two or more columns.
Do any of the versions support an XML data type Column, being read in?
Since its been a long time the PetaPoco is not updated and no one seems to work on it(and not accepting the patches), we have moved to OrmLite. Its really sad to see this awesome library left stagnant.
Is there a feature for batched updates? For example, do adds/edits/deletes on poco List<customer>, and then db.update<customer>(List) does some background voodoo to quickly apply all the changes to the database at once. My apps usually work on thousands of records at a time, and I would like to have adapter.update(datatable) efficiency but with Poco intellisense.
+100 vote for Composite Primary keys
+1 (Another vote for merging the fork from schotime's branch into V5 I see great extension from that fork and would definitely like to see those included in the new version) Main point of this branch btw is the support for composite primary keys
Awesome to see some updates to PetaPoco! We have been using it in production now for some time and I love it! Of course I have hacked it up to do my own thing, but mostly layered our code on top of the PetaPoco code where possible, but have still made some changes to the core. I am going to work on building the latest code and merging all my changes into the V5 tree, in my own GitHub brand and then sending a pull request back to Brad. I think some of my changes might be really useful for some people, such as our improved SQL select generation classes.
Great work! A Nuget Package for v5 would be much appreciated.
I like petapoco very much,more than Drapper
Thank you for updating the Nuget package. If you are looking for the next "great" feature for PetaPoco (or NPoco, which is a better, more acceptable name, at least in the US), would you consider building a POCO generator that uses all a database's stored procedures result sets as a basis? Most corporate environments demand the use of SP's.
@Guy - The ability to generate c# functions from the MS SQL SPs are part of ServiceStack Ormlite . Since Ormlite 's T4 file is a fork of PetaPoco, you can extract the code from it and merge it to the PetaPoco T4 file.
When I install the current Nuget package (which says 5.01), this is still a single file. Guess I'm confused on what the exact status of the "version 5" this blog post talks about is.
Ok, reread and caught the part about munging everything back into a single file.