Added GSoC report #3

1 files changed, 80 insertions, 0 deletions

diff --git a/docs/gsoc/03-report.txt b/docs/gsoc/03-report.txt
new file mode 100644
index 0000000..24d9ae4
--- /dev/null
+++ b/docs/gsoc/03-report.txt
@@ -0,0 +1,80 @@
+This is a progress report #3 for Project Grumpy.
+
+Now, since report two, there has been a big change of focus in the course of
+development, which means that we decided to drop our beloved and also greatly
+hated NoSQL approach (MongoDB) and instead go forward using regular RDBMS
+which in our case is good old PostgreSQL.
+
+Although there were some compelling arguments (ease of use being my
+favorable) for MongoDB, the biggest nail in its coffing was its lack of
+"support" for it from Gentoo's infra team. For them it was just another
+application they would have to take care of and around interwebs there's lots
+of 'MongoDB ate my data' reports on how error-prone MongoDB actually is
+(although data volumes in most of these cases were so high, that I cannot
+really imagine Grumpy running into these problem). But I can really
+understand their concerns. Besides, if you take a look at list of commits in
+MongoDB's official development repository [1], you can see why people are a
+bit concerned ;)
+
+[1] http://github.com/mongodb/mongo/commits
+
+Therefore we switched over to PostgreSQL, using SQLAlchemy as a glue layer
+between the database and application. SQLAlchemy is a blessing because using
+its object relational model, you do not actually have to write any SQL (just
+take a peek in the 'grumpy_sync' utility).
+
+Progress so far
+===============
+
+So far I have implemented portage -> database sync utility that is used to
+keep database in sync with portage content. Although it seems to handle most
+of the various portage quirks (like package moves via 'profiles/sync'), it
+still might run into issues in some corner cases and there is also minimal
+error recovery: it is currently designed to crash with RuntimeError when it
+detects something out of ordinary.
+
+Of course, the data model is far from complete - no proper handling of
+keywords, and I do not even store ebuild depends, rdepends and licenses in
+database - mainly because I currently don't have any use cases for these.
+
+Syncer can be found under 'utils' directory in the project directory.
+
+Future plans
+============
+
+As model and controller are ready, next stop is to write rudimentary web app
+for browsing portage contents, so people can finally see that I actually
+haven't slacked all this time.. :)
+
+Also, during portage import I noticed some really simple QA issues like
+invalid herd names in 'metadata.xml'. Plan is to write a 'herdcheck' plugin
+and implement database storage for these QA issues. And as I cannot let
+anyone to simply write to database, I need to implement API to let plugins
+interact with app.
+
+Having API means that I can start integrating with other QA tools around
+there, mainly tinderbox.
+
+And finally, testing. I currently have simple doctesting and auditing (via
+PyFlakes) framework in place, but general unit testing is still missing.
+
+As you can see, I'm a bit lagging my proposed timeline - I still haven't
+actually started looking how to create the 30-day stabilisation and upstream
+version checkers, but hopefully I can pick up the speed because I can now say
+that I have passed the biggest hurdle.. :)
+
+And I have also dropped my 'secret agenda' of documenting my experience with
+NoSQL databases as a series of articles written during this project...
+
+Project info
+============
+
+Git repository of Grumpy repo is available from [2].
+
+[2] http://git.overlays.gentoo.org/gitweb/?p=proj/grumpy.git;a=summary
+
+Project's semi-official IRC channel is #gentoo-grumpy on Freenode network,
+if you run into troubles when testing out this project, then just ping me with
+a message.
+
+PS. Bonus points for those who noticed that I dropped 'weekly' ;)