GSoC Status: Week 14

We celebrated for the Mid Autumn Festival last week, it is one of the most important festival in China. People spent time fooling around with friends to enjoy the full moon and taste some delicious mooncakes, so did I. I can never resist the temptation of a cup of tea along with a piece of mooncake, yummy!

At the same time, the hooks section of the internal blog was finished, it is still not good enough and I will keep polishing it.

For the next week I will still spend my time on the unfinished blog and complete the final evaluation of GSoC 2013.

Read on →

GSoC Status: Week 13

This week I made some minor improvments to the Duda MariaDB package in order to shutdown client socket correctly if the database connection closes unexpectedly. Pagination had been added to the demo web services of both packages to make the examples more complete.

Meanwhile, the adventure of Monkey internals continued and I had finished two more sections of the blog (Scheduler and Clock), as I promised last week. If you find this interesting or want to make some contribution, please don’t hesitate to write me emails :–).

For the next week, the packages will start to be merged into the source base of duda, I will keep my eyes on that. Also, I will prepare for the hard ‘pencil down’ date, though no big changes will happen to the code bases, there might be room for tiny improvments. Last but not least, I will carry on with the internal blog, two more sections are expected to be finished by the end of next week.

Read on →

GSoC Status: Week 11-12

In the first half of week 11, I spent most of my time completing the documentation of the PostgreSQL package. You may check it out here. Bugs and typos were fixed for that package as well.

After that I travelled back home to take a short break and stay with my dear family. For this reason week 12 was mostly occupied by family and friends. It is always comfortable and enjoyable when you can kick back and talk about your recent status with them!

Before I went home, I intended to make some fixes and continue the blog post with the working box in my apartment via VPN. Unfortunately it didn’t work out and all I got at home was a pretty old machine which can hardly open more than 4 tabs when browsing web. That’s why this post came late.

For next week I will be improving the codebases, both bug fixing and documentation polishing, as the final evaluation is coming. And two more sections of the internal blog will be accomplished.

Read on →

GSoC Status: Week 10

This week the PostgreSQL package was completed, connection pooling support was successfully added. Meanwhile, the sibling functions that are used for querying and escaping string was also introduced into the package to provide more choice for the user.

In order to demonstrate the usage of the PostgreSQL package, I have created another duda web service duda_postgresql_demo on Github, if you ever used the mariadb_demo web service, you’ll find it pretty similar. Check it out for your interest.

From now on till the end of GSoC I will test the two packages I am in charge of and fix some potential bugs to make them more stable. The documentation of PostgreSQL package will be completed in no time. Last but not least, I shall continue the blog about Monkey internals that I haven’t finished yet.

Read on →

GSoC Status: Week 9

This week I am happy to announce the basic PostgreSQL package can pass the build process and integrate with Monkey. The asynchronous query handling have been added to the package and serveral bugs got fixed. I wrote some tests to make sure it really worked out. You may want to have a look at the package.

Another stuff worth to mention about is I got a bug report for my MariaDB package (thanks to C1H1O3 on the irc), it is a good sign that someone is evaluating and testing the package because that will make it more stable and robust.

I tried to reproduce the bug and traced the code with gdb, after several tests I figured out the MariaDB package seemed to work properly. I wondered it might be a bug from the Duda framework so I set up another tests without using MariaDB package and could still reproduce the bug. It turned out to be a bug hidden in Duda and I have reported it on the irc, edsiper will take care of it and I will see what I can help.

C1H1O3 also advised me to use configuration file for connection pool parameters, which seems like a good idea as we don’t need to rebuild the application if all we need is just to change the parameters.

Here are some tasks I will be hanlding for the next week: * add connection pooling support to the PostgreSQL package * as there are often more than one function to do the same thing in PostgreSQL client libray libpq, I will add the sibling functions to the APIs of package to make it more flexible.

Read on →

GSoC Status: Week 8

This week the draft APIs of PostgreSQL package has come out. You may check it out on my Github, feel free to leave me a comment.

The APIs are modeled after the previous MariaDB package. By doing so we can hide the differences between MariaDB and PostgreSQL from our users and let the packages deal with the details.

The asynchronous APIs of PostgreSQL provide a more flexible way to issue non-blocking connection and query, for example you can use a string array to specify the connection options you are interested and leave others as default. Currently I am working on integraing the APIs with the event handling module of Duda.

Next week I will mainly focus on the asynchronous handling implementation of the PostgreSQL package.

Read on →

GSoC Status: Week 7

This week my effort was mainly focused on developing a web service to demonstrate usage of the MariaDB package and completing the documentation. You may check it out here.

I was also getting myself familiar with the C library APIs of PostgreSQL, the community comes with comprehensive and clear documentation. For the next couple days, the documentation website will undoubtedly be my most visited. What’s more, the library comes with native non-blocking support. It rocks, elephant!

This morning I also got an email from Google that announced my successfully pass of the midterm evaluation. It is an amazing moment and I would like to share with those that ever helped and guided me.

For the next week, I will get my hands dirty in the Duda PostgreSQL package by designing the basic data structures and function prototypes.

Read on →

GSoC Status: Week 6

This week the Duda MariaDB package was completed. I spent some time on reimplementing the connection pooling due to my misunderstanding of it. The new version maintain a pool of connections that are either connected or connecting. And a pool is associated with a web service instead of a thread. The pool shall be defined as global in the web service, but due to the lack of isolate namespace there can’t be same pool names in differenct web services.

The documentation of MariaDB package was also updated to explain the usage of it. And for those who want to install this package, please refer to the Monkey fork of my Github. Clone that repo and checkout to mariadb branch, environment needed to build this package is set up for you. (Notice: you need cmake and libaio to make things work, so don’t forget to install them.) More information can be found here.

Currently I am working on an example to demonstrate the usage of this package. It will be finished very soon.

For next week I will make some potential improvemnts to MariaDB package, start to review the asynchronous APIs of PostgreSQL and do some experiments.

Read on →

GSoC Status: Week 5

For the past a few days of this week, I was integrating the source code that required to build libmysqlclient.a into MariaDB package. Meanwhile I added multiple statements query support and connection pooling support to MariaDB package. All the features in my proposal has been implemented and it is happy to announce this package is coming near to its completion.

integrate libmysqlclient.a

It took a bit longer than I expected it would be as MariaDB is such a huge project (totally 223MiB after extracting). I did spend some time to figure out the relation of different subparts and picked up those that are really necessary for building the static library.

The build tools used by MariaDB is cmake, I was not fimiliar with cmake so that took me some other time to learn some basic usage. I have consulted from blog series of MariaDB but it did not offer enough information. I turned to more comprehensive documentation, although not all the options are available on MariaDB(such as -DWITH_DEBUG), it did teach me how to use the same build options used by official MySQL release. ( with option -DCMAKE_CONFIG=mysql_release)

As there are so many build targets, I seeked help on irc #maria and some guy told me to use make mysqlclient if all I wanted is the client library.

After I got all the missing pieces of the puzzle together, it finally worked out. I have tested on two other Linux distribution to make sure it was not a pure accident. :-D (Note: remember to install cmake and libaio)

Read on →

GSoC Status: Week 4

This week my time was mostly occupied by completing the MariaDB package. For now, the basic version of MariaDB package is available, it can happily communicate with the MariaDB server on my Linux box. I wrote a trivial duda web service to test some common tasks when dealing with a relational database. During the implementation and test I did encounter certain bugs, some are very obscure but thank godness I finally fixed them. The current version is still not the final one, some APIs may be altered in the future and new features will be introduced into it. You can retrieve the code here for your own interest. :-D

During this week I also discussed some questions I was not sure about with the guy who implemented the asynchronous APIs for MariaDB(known as knielsen on #maria). He was nice and answered all my questions in detail. And my mentor, edsiper, who was busy lately, gave me some advices on how to deal with the duda apis. I wanna thanks both of them here!

Here are some tasks I will be working on for the next week:

  • write documentation to explain the design and usage of the MariaDB package.
  • do some code cleanup and fix some potential bugs.
  • write a more comprehensive example for demonstrating How-To use MariaDB package.
  • do some survey and try to add connection pool support for the package to reduce cost of connection setup and release.
  • add source code of libmariadbclient as a third party dependency because on some Linux distribution this library may still not come with the asynchronous APIs.
  • add mutli-statement query support for the package.
Read on →