I promised asyncio with async-await syntax, so here you have it :)
I fixed all bugs I could find (quite many to be more exact, which
is normally a good sign), and as a result I could run some programs
using asyncio with async-await syntax without any error, with the
same results cpython 3.5 would give.
I implemented all tests of cpython for the async-await syntax and
added some tests to check if everything works together well if
combined in a more complex program. It's all working now.
The GIL problem I described earlier was just an incompatibility of
asyncio with the PyPy interpreter (pyinteractive). The error does not
occur otherwise.
I have been working on the py3.5-async branch in the PyPy
repository lately and that is also where I did all my checks if
everything is working. I also merged all my work into the main py3.5
branch. This branch was really broken though, because there are some
major changes from py3k to py3.5. I fixed some things in there, and
the PyPy team managed to fix a lot of it as well. My mentor sometimes
took some time to get important things to work, so my workflow
wouldn't get interrupted. While I was working on py3.5-async, a lot
of fixes have been done on py3.5, changing the behaviour a bit and
(possibly) breaking some other things. I have not yet checked
everything of my proposal on this branch yet, but with some help I
think I managed to get everything working there as well. At least it
all looks very promising for now.
Next to that I also managed to do some fixes I could find in the
unpack feature of my proposal. There have been some special cases
which lead to errors, for example if a function looks like this: def
f(*args, **kwds), and a call like that: f(5,b=2,*[6,7]), I got a
TypeError saying “expected string, got list object”. The problem
here was that certain elements had a wrong order on the stack, so
pulling them normally would not work. I made a fix checking for that
case, there are probably better solutions but it seems to work for
now.
I will probably keep working on py3.5 for a little bit longer,
because I would like to do some more things with the work of my
proposal. It would be interesting to see a benchmark to compare the
speed of my work with cpython. Also there is a lot to be done on
py3.5 to work correctly, so I might want to check that as well.
Here is a link to all of my work:
https://bitbucket.org/pypy/pypy/commits/all?search=Raffael+Tfirst
My experience with GSOC 2016
It's really been great to work with professional people on such a
huge project. My mind was blown as I could witness how much I was
able to learn in this short time. The PyPy team was able to motivate
me to achieve the goal of my proposal and to get a lot more
interested in compiler construction than I have already been before.
I am glad I took the chance to participate in this years GSOC. Of
course my luck with having such a helpful and productive team on my
side is one of the main reasons why I enjoyed it so much.
Thursday, August 18, 2016
Thursday, August 4, 2016
Only bug fixes left!
All changes of cpython have been implemented in PyPy, so all that's left to do now is fixing some bugs. Some minor changes had to be done, because not everything of cpython has to be implemented in PyPy. For example, in cpython slots are used to check the existence of a function and then call it. The type of an object has the information of valid functions, stored as elements inside structs. Here's an example of how the __await__ function get's called in cpython:
ot = Py_TYPE(o);
if (ot->tp_as_async != NULL) {
getter = ot→tp_as_async→am_await;
}
if (getter != NULL) {
PyObject *res = (*getter)(o);
if (res != NULL) { … }
return res;
}
PyErr_Format(PyExc_TypeError,
"object %.100s can't be used in 'await' expression",
ot→tp_name);
return NULL;
This 'getter' directs to the am_await slot in typeobject.c. There, a lookup is done having '__await__' as parameter. If it exists, it gets called, an error is raised otherwise.
In PyPy all of this is way more simple. Practically I just replace the getter with the lookup for __await__. All I want to do is call the method __await__ if it exists, so that's all to it. My code now looks like this:
w_await = space.lookup(self, “__await__”)
if w_await is None: …
res = space.get_and_call_function(w_await, self)
if is not None: …
return res
I also fixed the _set_sentinel problem I wrote about in the last post. All dependency problems of other (not yet covered) Python features have been fixed. I can already execute simple programs, but as soon as it gets a little more complex and uses certain asyncio features, I get an error about the GIL (global interpreter lock):
“Fatal RPython error: a thread is trying to wait for the GIL, but the GIL was not initialized”
First I have to read some descriptions of the GIL, because I am not sure where this problem could come up in my case.
There are also many minor bugs at the moment. I already fixed one of the bigger ones which didn't allow async or await to be the name of variables. I also just learned that something I implemented does not work with RPython which I wasn't aware of. My mentor is helping me out with that.
I also have to write more tests, because they are the safest and fastest way to check for errors. There are a few things I didn't test enough, so I need to catch up on writing tests a bit.
Things are not going forward as fast as I would love it to, because I often get to completely new things which I need to study first (like the GIL in this case, or the memoryview objects from the last blog entry). But there really shouldn't be much left to do now until everything works, so I am pretty optimistic with the time I have left. If I strive to complete this task soon, I am positive my proposal will be successful.
Subscribe to:
Posts (Atom)