It's alive! - The final days of GSOC 2016

I promised asyncio with async-await syntax, so here you have it :)
I fixed all bugs I could find (quite many to be more exact, which is normally a good sign), and as a result I could run some programs using asyncio with async-await syntax without any error, with the same results cpython 3.5 would give.
I implemented all tests of cpython for the async-await syntax and added some tests to check if everything works together well if combined in a more complex program. It's all working now.
The GIL problem I described earlier was just an incompatibility of asyncio with the PyPy interpreter (pyinteractive). The error does not occur otherwise.
I have been working on the py3.5-async branch in the PyPy repository lately and that is also where I did all my checks if everything is working. I also merged all my work into the main py3.5 branch. This branch was really broken though, because there are some major changes from py3k to py3.5. I fixed some things in there, and the PyPy team managed to fix a lot of it as well. My mentor sometimes took some time to get important things to work, so my workflow wouldn't get interrupted. While I was working on py3.5-async, a lot of fixes have been done on py3.5, changing the behaviour a bit and (possibly) breaking some other things. I have not yet checked everything of my proposal on this branch yet, but with some help I think I managed to get everything working there as well. At least it all looks very promising for now.
Next to that I also managed to do some fixes I could find in the unpack feature of my proposal. There have been some special cases which lead to errors, for example if a function looks like this: def f(*args, **kwds), and a call like that: f(5,b=2,*[6,7]), I got a TypeError saying “expected string, got list object”. The problem here was that certain elements had a wrong order on the stack, so pulling them normally would not work. I made a fix checking for that case, there are probably better solutions but it seems to work for now.


I will probably keep working on py3.5 for a little bit longer, because I would like to do some more things with the work of my proposal. It would be interesting to see a benchmark to compare the speed of my work with cpython. Also there is a lot to be done on py3.5 to work correctly, so I might want to check that as well.



Here is a link to all of my work:
https://bitbucket.org/pypy/pypy/commits/all?search=Raffael+Tfirst



My experience with GSOC 2016
It's really been great to work with professional people on such a huge project. My mind was blown as I could witness how much I was able to learn in this short time. The PyPy team was able to motivate me to achieve the goal of my proposal and to get a lot more interested in compiler construction than I have already been before.
I am glad I took the chance to participate in this years GSOC. Of course my luck with having such a helpful and productive team on my side is one of the main reasons why I enjoyed it so much.

Only bug fixes left!

All changes of cpython have been implemented in PyPy, so all that's left to do now is fixing some bugs. Some minor changes had to be done, because not everything of cpython has to be implemented in PyPy. For example, in cpython slots are used to check the existence of a function and then call it. The type of an object has the information of valid functions, stored as elements inside structs. Here's an example of how the __await__ function get's called in cpython:
ot = Py_TYPE(o);
if (ot->tp_as_async != NULL) {
getter = ot→tp_as_async→am_await;
}
if (getter != NULL) {
PyObject *res = (*getter)(o);
if (res != NULL) { … }
return res;
}
PyErr_Format(PyExc_TypeError,
"object %.100s can't be used in 'await' expression",
ot→tp_name);
return NULL;

This 'getter' directs to the am_await slot in typeobject.c. There, a lookup is done having '__await__' as parameter. If it exists, it gets called, an error is raised otherwise.
In PyPy all of this is way more simple. Practically I just replace the getter with the lookup for __await__. All I want to do is call the method __await__ if it exists, so that's all to it. My code now looks like this:
w_await = space.lookup(self, “__await__”)
if w_await is None: …
res = space.get_and_call_function(w_await, self)
if is not None: …
return res


I also fixed the _set_sentinel problem I wrote about in the last post. All dependency problems of other (not yet covered) Python features have been fixed. I can already execute simple programs, but as soon as it gets a little more complex and uses certain asyncio features, I get an error about the GIL (global interpreter lock):
“Fatal RPython error: a thread is trying to wait for the GIL, but the GIL was not initialized”
First I have to read some descriptions of the GIL, because I am not sure where this problem could come up in my case.
There are also many minor bugs at the moment. I already fixed one of the bigger ones which didn't allow async or await to be the name of variables. I also just learned that something I implemented does not work with RPython which I wasn't aware of. My mentor is helping me out with that.
I also have to write more tests, because they are the safest and fastest way to check for errors. There are a few things I didn't test enough, so I need to catch up on writing tests a bit.


Things are not going forward as fast as I would love it to, because I often get to completely new things which I need to study first (like the GIL in this case, or the memoryview objects from the last blog entry). But there really shouldn't be much left to do now until everything works, so I am pretty optimistic with the time I have left. If I strive to complete this task soon, I am positive my proposal will be successful.