diff options
79 files changed, 1537 insertions, 1827 deletions
diff --git a/README.rst b/README.rst index 688d611683..0e449b873c 100644 --- a/README.rst +++ b/README.rst @@ -1,5 +1,5 @@ ===================================== -PyPy: Python in Python Implementation +PyPy: Python in Python Implementation ===================================== Welcome to PyPy! @@ -23,6 +23,7 @@ and send us feedback! the pypy-dev team <pypy-dev@python.org> + Building ======== diff --git a/ctypes_configure/doc/configure.txt b/ctypes_configure/doc/configure.txt index ab72678dd2..dc78e0cadd 100644 --- a/ctypes_configure/doc/configure.txt +++ b/ctypes_configure/doc/configure.txt @@ -19,6 +19,4 @@ installation usage ===== -`sample.py`_ explains in details how to use it. - -.. _`sample.py`: http://codespeak.net/svn/pypy/dist/ctypes_configure/doc/sample.py +:source:`sample.py <ctypes_configure/doc/sample.py>` explains in details how to use it. diff --git a/pypy/doc/__pypy__-module.rst b/pypy/doc/__pypy__-module.rst index d9cd3e08ff..5b27ff8eb4 100644 --- a/pypy/doc/__pypy__-module.rst +++ b/pypy/doc/__pypy__-module.rst @@ -1,16 +1,14 @@ -======================= The ``__pypy__`` module ======================= The ``__pypy__`` module is the main entry point to special features provided -by PyPy's standard interpreter. Its content depends on `configuration options`_ -which may add new functionality and functions whose existence or non-existence -indicates the presence of such features. +by PyPy's standard interpreter. Its content depends on :doc:`configuration options <config/index>` +which may add new functionality and functions whose existence or non-existence +indicates the presence of such features. -.. _`configuration options`: config/index.html Generally available functionality -================================= +--------------------------------- - ``internal_repr(obj)``: return the interpreter-level representation of an object. @@ -18,28 +16,22 @@ Generally available functionality It works like a simplified array of characters (actually, depending on the configuration the ``array`` module internally uses this). + Transparent Proxy Functionality -=============================== +------------------------------- -If `transparent proxies`_ are enabled (with :config:`objspace.std.withtproxy`) +If :ref:`transparent proxies <tproxy>` are enabled (with :config:`objspace.std.withtproxy`) the following functions are put into ``__pypy__``: - ``tproxy(typ, controller)``: Return something that looks like it is of type typ. Its behaviour is completely controlled by the controller. See the docs - about `transparent proxies`_ for detail. - + about :ref:`transparent proxies <tproxy>` for detail. - ``get_tproxy_controller(obj)``: If obj is really a transparent proxy, return its controller. Otherwise return None. -.. _`transparent proxies`: objspace-proxies.html#tproxy - Functionality available on py.py (not after translation) -======================================================== +-------------------------------------------------------- - ``isfake(obj)``: returns True if ``obj`` is faked. - - ``interp_pdb()``: start a pdb at interpreter-level. - - - diff --git a/pypy/doc/architecture.rst b/pypy/doc/architecture.rst index 8c58ef6e89..ab4ff425f2 100644 --- a/pypy/doc/architecture.rst +++ b/pypy/doc/architecture.rst @@ -6,6 +6,7 @@ Goals and Architecture Overview This document gives an overview of the goals and architecture of PyPy. See :doc:`getting started <getting-started>` for a practical introduction and starting points. + Mission statement ----------------- diff --git a/pypy/doc/clr-module.rst b/pypy/doc/clr-module.rst index 6f4094426b..1368655fd4 100644 --- a/pypy/doc/clr-module.rst +++ b/pypy/doc/clr-module.rst @@ -1,4 +1,3 @@ -=============================== The ``clr`` module for PyPy.NET =============================== @@ -8,8 +7,7 @@ still missing and its interface might change in next versions, but it's still useful to experiment a bit with PyPy.NET. PyPy.NET provides an import hook that lets you to import .NET namespaces -seamlessly as they were normal Python modules. Then, - +seamlessly as they were normal Python modules. Then, PyPY.NET native classes try to behave as much as possible in the "expected" way both for the developers used to .NET and for the ones used to Python. @@ -34,8 +32,9 @@ they exist in both worlds: Moreover, all the usual Python features such as bound and unbound methods are available as well. + Example of usage -================ +---------------- Here is an example of interactive session using the ``clr`` module:: @@ -52,8 +51,9 @@ Here is an example of interactive session using the ``clr`` module:: >>>> print obj.Count 3 + Conversion of parameters -======================== +------------------------ When calling a .NET method Python objects are converted to .NET objects. Lots of effort have been taken to make the conversion as @@ -68,11 +68,12 @@ some sort of collection. The opposite .NET to Python conversions happens for the values returned by the methods. Again, primitive types are converted in a -straightforward way; non-primitive types are wrapped in a Python object, +straightforward way; non-primitive types are wrapped in a Python object, so that they can be treated as usual. + Overload resolution -=================== +------------------- When calling an overloaded method, PyPy.NET tries to find the best overload for the given arguments; for example, consider the @@ -94,7 +95,7 @@ TypeError exception is raised. Generic classes -================ +---------------- Generic classes are fully supported. To instantiate a generic class, you need to use the ``[]`` notation:: @@ -115,7 +116,7 @@ to use the ``[]`` notation:: External assemblies and Windows Forms -===================================== +------------------------------------- By default, you can only import .NET namespaces that belongs to already loaded assemblies. To load additional .NET assemblies, you can use diff --git a/pypy/doc/coding-guide.rst b/pypy/doc/coding-guide.rst index e5358b8cec..bc87ac2e89 100644 --- a/pypy/doc/coding-guide.rst +++ b/pypy/doc/coding-guide.rst @@ -1,21 +1,17 @@ -==================================== Coding Guide -==================================== +============ .. contents:: This document describes coding requirements and conventions for working with the PyPy code base. Please read it carefully and ask back any questions you might have. The document does not talk -very much about coding style issues. We mostly follow `PEP 8`_ though. +very much about coding style issues. We mostly follow :pep:`8` though. If in doubt, follow the style that is already present in the code base. -.. _`PEP 8`: http://www.python.org/dev/peps/pep-0008/ - -.. _`RPython`: Overview and motivation -======================== +------------------------ We are writing a Python interpreter in Python, using Python's well known ability to step behind the algorithmic problems as a language. At first glance, @@ -25,7 +21,7 @@ larger goals. CPython vs. PyPy -------------------- +~~~~~~~~~~~~~~~~ Compared to the CPython implementation, Python takes the role of the C Code. We rewrite the CPython interpreter in Python itself. We could @@ -48,16 +44,16 @@ but let's stick with this somewhat canonical approach. .. _interpreter-level: Application-level and interpreter-level execution and objects -------------------------------------------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Since Python is used for implementing all of our code base, there is a -crucial distinction to be aware of: that between *interpreter-level* objects and +crucial distinction to be aware of: that between *interpreter-level* objects and *application-level* objects. The latter are the ones that you deal with when you write normal python programs. Interpreter-level code, however, cannot invoke operations nor access attributes from application-level objects. You will immediately recognize any interpreter level code in PyPy, because half the variable and object names start with a ``w_``, which -indicates that they are `wrapped`_ application-level values. +indicates that they are `wrapped`_ application-level values. Let's show the difference with a simple example. To sum the contents of two variables ``a`` and ``b``, one would write the simple application-level @@ -80,10 +76,10 @@ from failures appearing in a python application level program that we are interpreting. -.. _`app-preferable`: +.. _app-preferable: -Application level is often preferable -------------------------------------- +Application level is often preferable +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Application-level code is substantially higher-level, and therefore correspondingly easier to write and debug. For example, suppose we want @@ -113,11 +109,11 @@ something much lower-level and involved, say something like:: space.setitem(w_self, w_key, w_value) This interpreter-level implementation looks much more similar to the C -source code. It is still more readable than its C counterpart because -it doesn't contain memory management details and can use Python's native -exception mechanism. +source code. It is still more readable than its C counterpart because +it doesn't contain memory management details and can use Python's native +exception mechanism. -In any case, it should be obvious that the application-level implementation +In any case, it should be obvious that the application-level implementation is definitely more readable, more elegant and more maintainable than the interpreter-level one (and indeed, dict.update is really implemented at applevel in PyPy). @@ -129,10 +125,11 @@ level of the object space before they can be executed), application level code is usually preferable. We have an abstraction (called the 'Gateway') which allows the caller of a function to remain ignorant of whether a particular function is implemented at application or -interpreter level. +interpreter level. + Our runtime interpreter is "RPython" ----------------------------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ In order to make a C code generator feasible all code on interpreter level has to restrict itself to a subset of the Python language, and we adhere to some @@ -173,14 +170,13 @@ enables the code generator to emit efficient machine level replacements for pure integer objects, for instance. -.. _`wrapping rules`: -.. _`wrapped`: +.. _wrapped: Wrapping rules -============== +-------------- Wrapping ---------- +~~~~~~~~ PyPy is made of Python source code at two levels: there is on the one hand *application-level code* that looks like normal Python code, and that @@ -207,7 +203,7 @@ with suitable interpreter-level classes with some amount of internal structure. For example, an application-level Python ``list`` -is implemented by the `standard object space`_ as an +is implemented by the :ref:`standard object space <standard-object-space>` as an instance of ``W_ListObject``, which has an instance attribute ``wrappeditems`` (an interpreter-level list which contains the application-level list's items as wrapped objects). @@ -216,7 +212,7 @@ The rules are described in more details below. Naming conventions ------------------- +~~~~~~~~~~~~~~~~~~ * ``space``: the object space is only visible at interpreter-level code, where it is by convention passed around by the name @@ -236,14 +232,14 @@ Naming conventions Operations on ``w_xxx`` ------------------------ +~~~~~~~~~~~~~~~~~~~~~~~ The core bytecode interpreter considers wrapped objects as black boxes. It is not allowed to inspect them directly. The allowed operations are all implemented on the object space: they are called ``space.xxx()``, where ``xxx`` is a standard operation name (``add``, ``getattr``, ``call``, ``eq``...). They are documented in the -`object space document`_. +:ref:`object space document <objspace-interface>`. A short warning: **don't do** ``w_x == w_y`` or ``w_x is w_y``! rationale for this rule is that there is no reason that two @@ -255,12 +251,11 @@ directly a interpreter-level bool. To check for identity, use ``space.is_true(space.is_(w_x, w_y))`` or better ``space.is_w(w_x, w_y)``. -.. _`object space document`: objspace.html#interface -.. _`applevel-exceptions`: +.. _applevel-exceptions: Application-level exceptions ----------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Interpreter-level code can use exceptions freely. However, all application-level exceptions are represented as an @@ -292,10 +287,10 @@ match an exception, as this will miss exceptions that are instances of subclasses. -.. _`modules`: +.. _modules: Modules in PyPy -=============== +--------------- Modules visible from application programs are imported from interpreter or application level files. PyPy reuses almost all python @@ -314,8 +309,9 @@ for the implementation. Note that there is no extra facility for pure-interpreter level modules, you just write a mixed module and leave the application-level part empty. + Determining the location of a module implementation ---------------------------------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ You can interactively find out where a module comes from, when running py.py. here are examples for the possible locations:: @@ -333,8 +329,9 @@ here are examples for the possible locations:: '/home/hpk/pypy-dist/lib-python/2.7/os.py' >>>> + Module directories / Import order ---------------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Here is the order in which PyPy looks up Python modules: @@ -357,10 +354,11 @@ Here is the order in which PyPy looks up Python modules: The modified CPython library. -.. _`modify modules`: + +.. _modify modules: Modifying a CPython library module or regression test -------------------------------------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Although PyPy is very compatible with CPython we sometimes need to change modules contained in our copy of the standard library, @@ -369,14 +367,15 @@ by default and CPython has a number of places where it relies on some classes being old-style. We just maintain those changes in place, -to see what is changed we have a branch called `vendot/stdlib` +to see what is changed we have a branch called `vendor/stdlib` wich contains the unmodified cpython stdlib -.. _`mixed module mechanism`: -.. _`mixed modules`: + +.. _mixed module mechanism: +.. _mixed-modules: Implementing a mixed interpreter/application level Module ---------------------------------------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ If a module needs to access PyPy's interpreter level then it is implemented as a mixed module. @@ -387,15 +386,13 @@ Only specified names will be exported to a Mixed Module's applevel namespace. Sometimes it is necessary to really write some functions in C (or -whatever target language). See `rffi`_ and `external functions -documentation`_ for details. The latter approach is cumbersome and +whatever target language). See :doc:`rffi <rpython:rffi>` and :ref:`external functions +documentation <rpython:extfunccalls>` for details. The latter approach is cumbersome and being phased out and former has currently quite a few rough edges. -.. _`rffi`: rffi.html -.. _`external functions documentation`: translation.html#extfunccalls application level definitions -............................. +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Application level specifications are found in the `appleveldefs` dictionary found in ``__init__.py`` files of directories in ``pypy/module``. @@ -410,8 +407,9 @@ The ``app_`` prefix indicates that the submodule ``app_inspect`` is interpreted at application level and the wrapped function value for ``locals`` will be extracted accordingly. + interpreter level definitions -............................. +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Interpreter level specifications are found in the ``interpleveldefs`` dictionary found in ``__init__.py`` files of directories in ``pypy/module``. @@ -431,7 +429,7 @@ the definition for ``operation.len()``:: return space.len(w_obj) Exposed interpreter level functions usually take a ``space`` argument -and some wrapped values (see `wrapping rules`_) . +and some wrapped values (see `Wrapping rules`_) . You can also use a convenient shortcut in ``interpleveldefs`` dictionaries: namely an expression in parentheses to specify an interpreter level @@ -450,8 +448,9 @@ creation of a new config option (such as --withmod-mymodule and --withoutmod-mymodule (the later being the default)) for py.py and translate.py. + Testing modules in ``lib_pypy/`` --------------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ You can go to the :source:`lib_pypy/pypy_test/` directory and invoke the testing tool ("py.test" or "python ../../pypy/test_all.py") to run tests against the @@ -461,26 +460,28 @@ and encouraged to let their tests run at interpreter level although This allows us to quickly test our python-coded reimplementations against CPython. + Testing modules in ``pypy/module`` ----------------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Simply change to ``pypy/module`` or to a subdirectory and `run the tests as usual`_. Testing modules in ``lib-python`` ------------------------------------ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ In order to let CPython's regression tests run against PyPy you can switch to the :source:`lib-python/` directory and run the testing tool in order to start compliance tests. (XXX check windows compatibility for producing test reports). + Naming conventions and directory layout -=========================================== +--------------------------------------- Directory and File Naming -------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~ - directories/modules/namespaces are always **lowercase** @@ -493,8 +494,9 @@ Directory and File Naming - keep filenames concise and completion-friendly. + Naming of python objects ------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~ - class names are **CamelCase** @@ -510,8 +512,9 @@ Naming of python objects includes w_self. Don't use ``w_`` in application level python only code. + Committing & Branching to the repository ------------------------------------------------------ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - write good log messages because several people are reading the diffs. @@ -522,22 +525,21 @@ Committing & Branching to the repository ``try1`` doesn't already exists) you should do:: hg branch try1 - + The branch will be recorded in the repository only after a commit. To switch back to the default branch:: - + hg update default - + For further details use the help or refer to the `official wiki`_:: - + hg help branch -.. _`official wiki`: http://mercurial.selenic.com/wiki/Branch +.. _official wiki: http://mercurial.selenic.com/wiki/Branch -.. _`using development tracker`: Using the development bug/feature tracker -========================================= +----------------------------------------- We have a `development tracker`_, based on Richard Jones' `roundup`_ application. You can file bugs, @@ -545,24 +547,23 @@ feature requests or see what's going on for the next milestone, both from an E-Mail and from a web interface. -.. _`development tracker`: https://bugs.pypy.org/ +.. _development tracker: https://bugs.pypy.org/ +.. _roundup: http://roundup.sourceforge.net/ + use your codespeak login or register ------------------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ If you have an existing codespeak account, you can use it to login within the tracker. Else, you can `register with the tracker`_ easily. - -.. _`register with the tracker`: https://bugs.pypy.org/user?@template=register -.. _`roundup`: http://roundup.sourceforge.net/ +.. _register with the tracker: https://bugs.pypy.org/user?@template=register -.. _`testing in PyPy`: -.. _`test-design`: +.. _testing: Testing in PyPy -=============== +--------------- Our tests are based on the `py.test`_ tool which lets you write unittests without boilerplate. All tests of modules @@ -575,12 +576,11 @@ basically two types of unit tests: - **Application Level tests**. They run at application level which means that they look like straight python code but they are interpreted by PyPy. -.. _`standard object space`: objspace.html#standard-object-space -.. _`objectspace`: objspace.html -.. _`py.test`: http://pytest.org/ +.. _py.test: http://pytest.org/ + Interpreter level tests ------------------------ +~~~~~~~~~~~~~~~~~~~~~~~ You can write test functions and methods like this:: @@ -596,8 +596,9 @@ classes is mandatory. In both cases you can import Python modules at module global level and use plain 'assert' statements thanks to the usage of the `py.test`_ tool. + Application Level tests ------------------------ +~~~~~~~~~~~~~~~~~~~~~~~ For testing the conformance and well-behavedness of PyPy it is often sufficient to write "normal" application-level @@ -632,10 +633,11 @@ via self (but without the ``w_``) in the actual test method. An example:: assert self.d["a"] == 1 assert self.d["b"] == 2 -.. _`run the tests as usual`: + +.. _run the tests as usual: Command line tool test_all --------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~ You can run almost all of PyPy's tests by invoking:: @@ -645,8 +647,9 @@ which is a synonym for the general `py.test`_ utility located in the ``py/bin/`` directory. For switches to modify test execution pass the ``-h`` option. + Coverage reports ----------------- +~~~~~~~~~~~~~~~~ In order to get coverage reports the `pytest-cov`_ plugin is included. it adds some extra requirements ( coverage_ and `cov-core`_ ) @@ -654,12 +657,13 @@ and can once they are installed coverage testing can be invoked via:: python test_all.py --cov file_or_direcory_to_cover file_or_directory -.. _`pytest-cov`: http://pypi.python.org/pypi/pytest-cov -.. _`coverage`: http://pypi.python.org/pypi/coverage -.. _`cov-core`: http://pypi.python.org/pypi/cov-core +.. _pytest-cov: http://pypi.python.org/pypi/pytest-cov +.. _coverage: http://pypi.python.org/pypi/coverage +.. _cov-core: http://pypi.python.org/pypi/cov-core + Test conventions ----------------- +~~~~~~~~~~~~~~~~ - adding features requires adding appropriate tests. (It often even makes sense to first write the tests so that you are sure that they @@ -669,29 +673,28 @@ Test conventions which contain unit tests. Such scripts can usually be executed directly or are collectively run by pypy/test_all.py -.. _`change documentation and website`: + +.. _change documentation and website: Changing documentation and website -================================== +---------------------------------- documentation/website files in your local checkout ---------------------------------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Most of the PyPy's documentation is kept in `pypy/doc`. You can simply edit or add '.rst' files which contain ReST-markuped files. Here is a `ReST quickstart`_ but you can also just look at the existing documentation and see how things work. -.. _`ReST quickstart`: http://docutils.sourceforge.net/docs/user/rst/quickref.html - Note that the web site of http://pypy.org/ is maintained separately. For now it is in the repository https://bitbucket.org/pypy/pypy.org -Automatically test documentation/website changes ------------------------------------------------- +.. _ReST quickstart: http://docutils.sourceforge.net/docs/user/rst/quickref.html + -.. _`sphinx home page`: -.. _`sphinx`: http://sphinx.pocoo.org/ +Automatically test documentation/website changes +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ We automatically check referential integrity and ReST-conformance. In order to run the tests you need sphinx_ installed. Then go to the local checkout @@ -710,3 +713,5 @@ the documentation issue:: make linkcheck which will check that remote URLs are reachable. + +.. _sphinx: http://sphinx.pocoo.org/ diff --git a/pypy/doc/conf.py b/pypy/doc/conf.py index 0e623c5e39..4067b878b9 100644 --- a/pypy/doc/conf.py +++ b/pypy/doc/conf.py @@ -195,7 +195,8 @@ latex_use_modindex = False # Example configuration for intersphinx: refer to the Python standard library. -intersphinx_mapping = {'http://docs.python.org/': None} +intersphinx_mapping = {'python': ('http://docs.python.org/2.7', None), + 'rpython': ('http://rpython.readthedocs.org/en/improve-docs/', None)} # -- Options for manpage output------------------------------------------------- diff --git a/pypy/doc/configuration.rst b/pypy/doc/configuration.rst index 32af2d9993..ede5396de6 100644 --- a/pypy/doc/configuration.rst +++ b/pypy/doc/configuration.rst @@ -1,4 +1,3 @@ -============================= PyPy's Configuration Handling ============================= @@ -6,16 +5,17 @@ Due to more and more available configuration options it became quite annoying to hand the necessary options to where they are actually used and even more annoying to add new options. To circumvent these problems configuration management was introduced. There all the necessary options are stored in a -configuration object, which is available nearly everywhere in the `RPython +configuration object, which is available nearly everywhere in the `RPython toolchain`_ and in the standard interpreter so that adding new options becomes trivial. Options are organized into a tree. Configuration objects can be created in different ways, there is support for creating an optparse command line parser automatically. -_`RPython toolchain`: translation.html +.. _RPython toolchain: https://rpython.readthedocs.org/ + Main Assumption -=============== +--------------- Configuration objects are produced at the entry points and handed down to where they are actually used. This keeps configuration local but available @@ -24,7 +24,7 @@ command line. API Details -=========== +----------- The handling of options is split into two parts: the description of which options are available, what their possible values and defaults are and how they @@ -40,8 +40,9 @@ The Options are organized in a tree. Every option has a name, as does every option group. The parts of the full name of the option are separated by dots: e.g. ``config.translation.thread``. + Description of Options ----------------------- +~~~~~~~~~~~~~~~~~~~~~~ All the constructors take a ``name`` and a ``doc`` argument as first arguments to give the option or option group a name and to document it. Most constructors @@ -70,6 +71,7 @@ This class is used to group suboptions. ``children`` is a list of option descriptions (including ``OptionDescription`` instances for nested namespaces). + ``ChoiceOption`` ++++++++++++++++ @@ -81,10 +83,11 @@ Represents a choice out of several objects. The option can also have the value ``requires`` is a dictionary mapping values to lists of of two-element tuples. + ``BoolOption`` ++++++++++++++ -Represents a choice between ``True`` and ``False``. +Represents a choice between ``True`` and ``False``. ``__init__(self, name, doc, default=None, requires=None, suggests=None, cmdline=DEFAULT, negation=True)`` ``default`` specifies the default value of the option. ``requires`` is @@ -103,7 +106,6 @@ Represents a choice between ``True`` and ``False``. Represents a choice of an integer. ``__init__(self, name, doc, default=None, cmdline=DEFAULT)`` - ``FloatOption`` @@ -112,7 +114,6 @@ Represents a choice of an integer. Represents a choice of a floating point number. ``__init__(self, name, doc, default=None, cmdline=DEFAULT)`` - ``StrOption`` @@ -121,12 +122,10 @@ Represents a choice of a floating point number. Represents the choice of a string. ``__init__(self, name, doc, default=None, cmdline=DEFAULT)`` - - Configuration Objects ---------------------- +~~~~~~~~~~~~~~~~~~~~~ ``Config`` objects hold the chosen values for the options (of the default, if no choice was made). A ``Config`` object is described by an @@ -164,7 +163,7 @@ Description of the (useful) methods on ``Config``: Production of optparse Parsers ------------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To produce an optparse parser use the function ``to_optparse``. It will create an option parser using callbacks in such a way that the config object used for @@ -182,10 +181,9 @@ creating the parser is updated automatically. The usage of config objects in PyPy -=================================== +----------------------------------- -The two large parts of PyPy, the Python interpreter and the `RPython -toolchain`_ +The two large parts of PyPy, the Python interpreter and the RPython toolchain, have two separate sets of options. The translation toolchain options can be found on the ``config`` attribute of all ``TranslationContext`` instances and are described in :source:`rpython/config/translationoption.py`. The interpreter options diff --git a/pypy/doc/contributor.rst b/pypy/doc/contributor.rst index 53501fe192..74b799fb02 100644 --- a/pypy/doc/contributor.rst +++ b/pypy/doc/contributor.rst @@ -1,4 +1,3 @@ - Contributors to PyPy ==================== diff --git a/pypy/doc/cppyy.rst b/pypy/doc/cppyy.rst index 854fd8b3db..54fa7fce70 100644 --- a/pypy/doc/cppyy.rst +++ b/pypy/doc/cppyy.rst @@ -1,4 +1,3 @@ -============================ cppyy: C++ bindings for PyPy ============================ @@ -13,11 +12,11 @@ of C++11. The work on the cling backend has so far been done only for CPython, but bringing it to PyPy is a lot less work than developing it in the first place. -.. _`Reflex`: http://root.cern.ch/drupal/content/reflex -.. _`CINT`: http://root.cern.ch/drupal/content/cint -.. _`cling`: http://root.cern.ch/drupal/content/cling -.. _`llvm`: http://llvm.org/ -.. _`clang`: http://clang.llvm.org/ +.. _Reflex: http://root.cern.ch/drupal/content/reflex +.. _CINT: http://root.cern.ch/drupal/content/cint +.. _cling: http://root.cern.ch/drupal/content/cling +.. _llvm: http://llvm.org/ +.. _clang: http://clang.llvm.org/ This document describes the version of cppyy that lives in the main branch of PyPy. @@ -25,7 +24,7 @@ The development of cppyy happens in the "reflex-support" branch. Motivation -========== +---------- To provide bindings to another language in CPython, you program to a generic C-API that exposes many of the interpreter features. @@ -57,11 +56,11 @@ As another example, cling allows automatic instantiations of templates. See this description of the `cppyy architecture`_ for further details. -.. _`cppyy architecture`: http://morepypy.blogspot.com/2012/06/architecture-of-cppyy.html +.. _cppyy architecture: http://morepypy.blogspot.com/2012/06/architecture-of-cppyy.html Installation -============ +------------ There are two ways of using cppyy, and the choice depends on how pypy-c was built: the backend can be builtin, or dynamically loadable. @@ -72,9 +71,7 @@ That advantage will disappear over time, however, with improvements in the JIT. Therefore, this document assumes that the dynamically loadable backend is chosen (it is, by default). -See the `backend documentation`_. - -.. _`backend documentation`: cppyy_backend.html +See the :doc:`backend documentation <cppyy_backend>`. A standalone version of Reflex that also provides the dynamically loadable backend is available for `download`_. @@ -83,8 +80,8 @@ comes with `ROOT`_, which may be part of your Linux distribution as part of the selection of scientific software) will also work for a build with the builtin backend. -.. _`download`: http://cern.ch/wlav/reflex-2013-04-23.tar.bz2 -.. _`ROOT`: http://root.cern.ch/ +.. _download: http://cern.ch/wlav/reflex-2013-04-23.tar.bz2 +.. _ROOT: http://root.cern.ch/ Besides Reflex, you probably need a version of `gccxml`_ installed, which is most easily provided by the packager of your system. @@ -94,7 +91,7 @@ That's why the medium term plan is to move to cling. Note that gccxml is only needed to generate reflection libraries. It is not needed to use them. -.. _`gccxml`: http://www.gccxml.org +.. _gccxml: http://www.gccxml.org To install the standalone version of Reflex, after download:: @@ -112,8 +109,8 @@ If you downloaded or built the whole of ROOT, ``REFLEXHOME`` should be equal to ``ROOTSYS``. The following is optional, and is only to show how pypy-c can be build -`from source`_, for example to get at the main development branch of cppyy. -The `backend documentation`_ has more details on the backend-specific +:ref:`from source <translate-pypy>`, for example to get at the main development branch of cppyy. +The :doc:`backend documentation <cppyy_backend>` has more details on the backend-specific prerequisites. Then run the translation to build ``pypy-c``:: @@ -121,7 +118,7 @@ Then run the translation to build ``pypy-c``:: $ hg clone https://bitbucket.org/pypy/pypy $ cd pypy $ hg up reflex-support # optional - + # This example shows python, but using pypy-c is faster and uses less memory $ python rpython/translator/goal/translate.py --opt=jit pypy/goal/targetpypystandalone --withmod-cppyy @@ -129,15 +126,12 @@ This will build a ``pypy-c`` that includes the cppyy module, and through that, Reflex support. Of course, if you already have a pre-built version of the ``pypy`` interpreter, you can use that for the translation rather than ``python``. -If not, you may want `to obtain a binary distribution`_ to speed up the +If not, you may want :ref:`to obtain a binary distribution <prebuilt-pypy>` to speed up the translation step. -.. _`from source`: https://bitbucket.org/pypy/pypy/overview -.. _`to obtain a binary distribution`: http://doc.pypy.org/en/latest/getting-started.html#download-a-pre-built-pypy - Basic bindings example -====================== +---------------------- Now test with a trivial example whether all packages are properly installed and functional. @@ -186,7 +180,7 @@ That's all there is to it! Automatic class loader -====================== +---------------------- There is one big problem in the code above, that prevents its use in a (large scale) production setting: the explicit loading of the reflection library. @@ -234,7 +228,7 @@ As a caveat, note that the class loader is currently limited to classes only. Advanced example -================ +---------------- The following snippet of C++ is very contrived, to allow showing that such pathological code can be handled and to show how certain features play out in @@ -306,7 +300,7 @@ For our purposes, the following rather straightforward selection will do <function name="BaseFactory" /> </lcgdict> -.. _`selection file`: http://root.cern.ch/drupal/content/generating-reflex-dictionaries +.. _selection file: http://root.cern.ch/drupal/content/generating-reflex-dictionaries Now the reflection info can be generated and compiled:: @@ -366,14 +360,12 @@ It is not always possible to provide exact mapping between python and C++ (active memory management is one such case), but by and large, if the use of a feature does not strike you as obvious, it is more likely to simply be a bug. That is a strong statement to make, but also a worthy goal. -For the C++ side of the examples, refer to this `example code`_, which was +For the C++ side of the examples, refer to this :doc:`example code <cppyy_example>`, which was bound using:: $ genreflex example.h --deep --rootmap=libexampleDict.rootmap --rootmap-lib=libexampleDict.so $ g++ -fPIC -rdynamic -O2 -shared -I$REFLEXHOME/include example_rflx.cpp -o libexampleDict.so -L$REFLEXHOME/lib -lReflex -.. _`example code`: cppyy_example.html - * **abstract classes**: Are represented as python classes, since they are needed to complete the inheritance hierarchies, but will raise an exception if an attempt is made to instantiate from them. @@ -402,7 +394,7 @@ bound using:: >>>> c = ConcreteClass() >>>> c.array_method(array('d', [1., 2., 3., 4.]), 4) 1 2 3 4 - >>>> + >>>> * **builtin data types**: Map onto the expected equivalent python types, with the caveat that there may be size differences, and thus it is possible that @@ -446,7 +438,7 @@ bound using:: >>>> n = Namespace.ConcreteClass.NestedClass() >>>> type(n) <class '__main__.Namespace::ConcreteClass::NestedClass'> - >>>> + >>>> * **data members**: Public data members are represented as python properties and provide read and write access on instances as expected. @@ -483,7 +475,7 @@ bound using:: >>>> print ConcreteClass.array_method.__doc__ void ConcreteClass::array_method(int*, int) void ConcreteClass::array_method(double*, int) - >>>> + >>>> * **enums**: Are translated as ints with no further checking. @@ -511,9 +503,9 @@ bound using:: | cppyy.CPPObject | __builtin__.CPPInstance | __builtin__.object - | + | | Methods defined here: - | + | | ConcreteClass(self, *args) | ConcreteClass::ConcreteClass(const ConcreteClass&) | ConcreteClass::ConcreteClass(int) @@ -531,7 +523,7 @@ bound using:: >>>> c = ConcreteClass() >>>> c._python_owns # True: object created in Python True - >>>> + >>>> * **methods**: Are represented as python methods and work as expected. They are first class objects and can be bound to an instance. @@ -566,7 +558,7 @@ bound using:: >>>> from cppyy.gbl import ConcreteClass >>>> print ConcreteClass() Hello operator const char*! - >>>> + >>>> * **operator overloads**: If defined in the C++ class and if a python equivalent is available (not always the case, think e.g. of ``operator||``), @@ -618,7 +610,7 @@ bound using:: <class '__main__.std::vector<int>'> >>>> cppyy.gbl.std.vector(int)() # instantiates class -> object <__main__.std::vector<int> object at 0x00007fe480ba4bc0> - >>>> + >>>> Note that templates can be build up by handing actual types to the class instantiation (as done in this vector example), or by passing in the list of @@ -795,8 +787,8 @@ The rest works the same way: the fast path will be used transparently (which also means that you can't actually find out whether it is in use, other than by running a micro-benchmark or a JIT test). -.. _`provided`: http://cern.ch/wlav/reflex-2013-04-23.tar.bz2 -.. _`genreflex-methptrgetter.patch`: https://bitbucket.org/pypy/pypy/src/default/pypy/module/cppyy/genreflex-methptrgetter.patch +.. _provided: http://cern.ch/wlav/reflex-2013-04-23.tar.bz2 +.. _genreflex-methptrgetter.patch: https://bitbucket.org/pypy/pypy/src/default/pypy/module/cppyy/genreflex-methptrgetter.patch CPython ======= @@ -808,7 +800,7 @@ If you installed ROOT, rather than just Reflex, PyCintex should be available immediately if you add ``$ROOTSYS/lib`` to the ``PYTHONPATH`` environment variable. -.. _`PyROOT`: http://root.cern.ch/drupal/content/pyroot +.. _PyROOT: http://root.cern.ch/drupal/content/pyroot There are a couple of minor differences between PyCintex and cppyy, most to do with naming. @@ -861,4 +853,4 @@ the bindings module, be it ``cppyy`` or ``libPyROOT.so`` (in PyCintex). Although ``genreflex`` is indeed a Python tool, the generated reflection information is completely independent of Python. -.. _`Py3k`: https://bitbucket.org/pypy/pypy/src/py3k +.. _Py3k: https://bitbucket.org/pypy/pypy/src/py3k diff --git a/pypy/doc/cppyy_backend.rst b/pypy/doc/cppyy_backend.rst index 7ad46360b2..694d26b797 100644 --- a/pypy/doc/cppyy_backend.rst +++ b/pypy/doc/cppyy_backend.rst @@ -1,26 +1,22 @@ -================== Backends for cppyy ================== The cppyy module needs a backend to provide the C++ reflection information on which the Python bindings are build. The backend is called through a C-API, which can be found in the PyPy sources -in: `pypy/module/cppyy/include/capi.h`_. +in: :source:`pypy/module/cppyy/include/capi.h`. There are two kinds of API calls: querying about reflection information, which are used during the creation of Python-side constructs, and making the actual calls into C++. The objects passed around are all opaque: cppyy does not make any assumptions about them, other than that the opaque handles can be copied. Their definition, however, appears in two places: in the C code (in capi.h), -and on the RPython side (in `capi_types.py`_), so if they are changed, they +and on the RPython side (in :source:`capi_types.py <pypy/module/cppyy/capi/capi_types.py>`), so if they are changed, they need to be changed on both sides. -.. _`pypy/module/cppyy/include/capi.h`: https://bitbucket.org/pypy/pypy/src/default/pypy/module/cppyy/include/capi.h -.. _`capi_types.py`: https://bitbucket.org/pypy/pypy/src/default/pypy/module/cppyy/capi/capi_types.py - There are two places where selections in the RPython code affect the choice (and use) of the backend. -The first is in `pypy/module/cppyy/capi/__init__.py`_:: +The first is in :source:`pypy/module/cppyy/capi/__init__.py`:: # choose C-API access method: from pypy.module.cppyy.capi.loadable_capi import * @@ -29,19 +25,15 @@ The first is in `pypy/module/cppyy/capi/__init__.py`_:: The default is the loadable C-API. Comment it and uncomment the builtin C-API line, to use the builtin version. -.. _`pypy/module/cppyy/capi/__init__.py`: https://bitbucket.org/pypy/pypy/src/default/pypy/module/cppyy/capi/__init__.py - Next, if the builtin C-API is chosen, the specific backend needs to be set as well (default is Reflex). -This second choice is in `pypy/module/cppyy/capi/builtin_capi.py`_:: +This second choice is in :source:`pypy/module/cppyy/capi/builtin_capi.py`:: import reflex_capi as backend #import cint_capi as backend After those choices have been made, built pypy-c as usual. -.. _`pypy/module/cppyy/capi/builtin_capi.py`: https://bitbucket.org/pypy/pypy/src/default/pypy/module/cppyy/capi/builtin_capi.py - When building pypy-c from source, keep the following in mind. If the loadable_capi is chosen, no further prerequisites are needed. However, for the build of the builtin_capi to succeed, the ``ROOTSYS`` diff --git a/pypy/doc/cpython_differences.rst b/pypy/doc/cpython_differences.rst index 52f07555f5..1355afb0f7 100644 --- a/pypy/doc/cpython_differences.rst +++ b/pypy/doc/cpython_differences.rst @@ -1,4 +1,3 @@ -==================================== Differences between PyPy and CPython ==================================== @@ -19,21 +18,21 @@ List of extension modules that we support: * Supported as built-in modules (in :source:`pypy/module/`): __builtin__ - `__pypy__`_ + :doc:`__pypy__ <__pypy__-module>` _ast _codecs _collections - `_continuation`_ - `_ffi`_ + :doc:`_continuation <stackless>` + :doc:`_ffi <ctypes-implementation>` _hashlib _io _locale _lsprof _md5 - `_minimal_curses`_ + :doc:`_minimal_curses <config/objspace.usemodules._minimal_curses>` _multiprocessing _random - `_rawffi`_ + :doc:`_rawffi <ctypes-implementation>` _sha _socket _sre @@ -95,13 +94,7 @@ The extension modules (i.e. modules written in C, in the standard CPython) that are neither mentioned above nor in :source:`lib_pypy/` are not available in PyPy. (You may have a chance to use them anyway with `cpyext`_.) -.. the nonstandard modules are listed below... -.. _`__pypy__`: __pypy__-module.html -.. _`_continuation`: stackless.html -.. _`_ffi`: ctypes-implementation.html -.. _`_rawffi`: ctypes-implementation.html -.. _`_minimal_curses`: config/objspace.usemodules._minimal_curses.html -.. _`cpyext`: http://morepypy.blogspot.com/2010/04/using-cpython-extension-modules-with.html +.. _cpyext: http://morepypy.blogspot.com/2010/04/using-cpython-extension-modules-with.html Differences related to garbage collection strategies @@ -213,6 +206,7 @@ but ``foo`` on CPython:: >>>> print d1['a'] 42 + Mutating classes of objects which are already used as dictionary keys --------------------------------------------------------------------- @@ -260,6 +254,7 @@ Unless this behavior is clearly present by design and documented as such (as e.g. for hasattr()), in most cases PyPy lets the exception propagate instead. + Object Identity of Primitive Values, ``is`` and ``id`` ------------------------------------------------------- diff --git a/pypy/doc/ctypes-implementation.rst b/pypy/doc/ctypes-implementation.rst index aa4b444f5e..d279dffc90 100644 --- a/pypy/doc/ctypes-implementation.rst +++ b/pypy/doc/ctypes-implementation.rst @@ -1,9 +1,8 @@ -============================ PyPy's ctypes implementation ============================ Summary -======== +-------- Terminology: @@ -28,10 +27,11 @@ We reused the ``ctypes`` package version 1.0.2 as-is from CPython. We implemented ``_ctypes`` which is a C module in CPython mostly in pure Python based on a lower-level layer extension module ``_rawffi``. -.. _`libffi`: http://sources.redhat.com/libffi/ +.. _libffi: http://sources.redhat.com/libffi/ + Low-level part: ``_rawffi`` -============================ +--------------------------- This PyPy extension module (:source:`pypy/module/_rawffi`) exposes a simple interface to create C objects (arrays and structures) and calling functions @@ -44,15 +44,17 @@ We tried to keep this module as small as possible. It is conceivable that other implementations (e.g. Jython) could use our ctypes implementation by writing their version of ``_rawffi``. + High-level parts -================= +----------------- The reused ``ctypes`` package lives in ``lib_pypy/ctypes``. ``_ctypes`` implementing the same interface as ``_ctypes`` in CPython is in ``lib_pypy/_ctypes``. + Discussion and limitations -============================= +----------------------------- Reimplementing ctypes features was in general possible. PyPy supports pluggable garbage collectors, some of them are moving collectors, this @@ -90,13 +92,15 @@ current implementation: between its primitive types and user subclasses of its primitive types + Running application examples -============================== +------------------------------ `pyglet`_ is known to run. We also had some success with pygame-ctypes (which is no longer maintained) and with a snapshot of the experimental pysqlite-ctypes. We will only describe how to run the pyglet examples. + pyglet -------- +~~~~~~ We tried pyglet checking it out from its repository at revision 1984. @@ -131,17 +135,17 @@ We did not try the following examples: - video.py needs avbin - soundscape needs avbin -.. _`pyglet`: http://pyglet.org/ +.. _pyglet: http://pyglet.org/ ctypes configure -================= +----------------- We also released ``ctypes-configure``, which is an experimental package trying to approach the portability issues of ctypes-based code. idea ----- +~~~~ One of ctypes problems is that ctypes programs are usually not very platform-independent. We created ctypes_configure, which invokes c @@ -150,11 +154,11 @@ exact sizes of types (for example size_t), ``#defines``, exact outline of structures etc. It replaces in this regard code generator (h2py). installation ------------- +~~~~~~~~~~~~ ``easy_install ctypes_configure`` usage ------ +~~~~~ :source:`ctypes_configure/doc/sample.py` explains in details how to use it. diff --git a/pypy/doc/dev_method.rst b/pypy/doc/dev_method.rst index c6adfcaa1a..7b5eb89437 100644 --- a/pypy/doc/dev_method.rst +++ b/pypy/doc/dev_method.rst @@ -22,7 +22,7 @@ Main tools for achieving this is: * py.test - automated testing * Mercurial - version control * Transparent communication and documentation (mailinglists, IRC, tutorials - etc etc) + etc etc) Sprint driven development: @@ -33,7 +33,7 @@ What is a sprint and why are we sprinting? Originally the sprint methodology used in the Python community grew from practices within Zope3 development. The definition of a sprint is "two-day or three-day focused development session, in which developers pair off together -in a room and focus on building a particular subsystem". +in a room and focus on building a particular subsystem". Other typical sprint factors: @@ -91,7 +91,7 @@ during conferences like PyCon and EuroPython, the team will have an easier task of recruiting new talents to the team. It also vitalizes the community and increases the contact between the different Python implementation projects. - + As always with methodologies you have to adapt them to fit your project (and not the other way around which is much too common). The PyPy team have been sprinting since early 2003 and have done 22 sprints so far, 19 in Europe, 2 @@ -100,7 +100,7 @@ team and those are the one we are summarizing here. How is it done? -+++++++++++++++ +~~~~~~~~~~~~~~~ There are several aspects of a sprint. In the PyPy team we focus on: 1. Content (goal) @@ -171,7 +171,7 @@ There are several aspects of a sprint. In the PyPy team we focus on: (or even better: in between so that the memory is fresh) a sprint report is written by one of the developers and updated to codespeak, this is a kind of summary of the entire sprint and it tells of the work done and the - people involved. + people involved. One very important strategy when planning the venue is cost efficiency. Keeping accommodation and food/travel costs as low as possible @@ -181,17 +181,17 @@ There are several aspects of a sprint. In the PyPy team we focus on: (travel expenses and accommodation) and because most of the funding is so called matched funding we pay for most of our expenses in our own organizations and companies anyway. - + 4. Process - a typical PyPy sprint is 7 days with a break day in the middle. Usually sprinters show up the day before the sprint starts. The - first day has a start up meeting, with tutorials if there are participants + first day has a start up meeting, with tutorials if there are participants new to the project or if some new tool or feature have been implemented. A short presentation of the participants and their background and expectations is also good to do. Unfortunately there is always time spent the first day, mostly in the morning when people arrive to get the internet and server infrastructure up and running. That is why we are, through - documentation_, trying to get participants to set up the tools and + :doc:`documentation <getting-started>`, trying to get participants to set up the tools and configurations needed before they arrive to the sprint. Approximate hours being held are 10-17, but people tend to stay longer to @@ -224,10 +224,8 @@ There are several aspects of a sprint. In the PyPy team we focus on: some middle ground - thus increases the importance of feedback. -.. _documentation: getting-started.html - Can I join in? -++++++++++++++ +~~~~~~~~~~~~~~ Of course. Just follow the work on pypy-dev and if you specifically are interested in information about our sprints - subscribe to @@ -236,4 +234,3 @@ pypy-sprint@codespeak.net and read the news on codespeak for announcements etc. If you think we should sprint in your town - send us an email - we are very interested in using sprints as away of making contact with active developers (Python/compiler design etc)! - diff --git a/pypy/doc/dir-reference.rst b/pypy/doc/dir-reference.rst index 01119560ea..efa1f0d083 100644 --- a/pypy/doc/dir-reference.rst +++ b/pypy/doc/dir-reference.rst @@ -1,8 +1,8 @@ -PyPy directory cross-reference ------------------------------- +PyPy directory cross-reference +============================== -Here is a fully referenced alphabetical two-level deep -directory overview of PyPy: +Here is a fully referenced alphabetical two-level deep +directory overview of PyPy: ======================================== ============================================ Directory explanation/links @@ -21,120 +21,41 @@ Directory explanation/links :source:`pypy/doc/discussion/` drafts of ideas and documentation -``doc/*/`` other specific documentation topics or tools +:source:`pypy/goal/` our :ref:`main PyPy-translation scripts <translate-pypy>` + live here -:source:`pypy/interpreter/` `bytecode interpreter`_ and related objects - (frames, functions, modules,...) +:source:`pypy/interpreter/` :doc:`bytecode interpreter <interpreter>` and related objects + (frames, functions, modules,...) :source:`pypy/interpreter/pyparser/` interpreter-level Python source parser :source:`pypy/interpreter/astcompiler/` interpreter-level bytecode compiler, via an AST representation -:source:`pypy/module/` contains `mixed modules`_ - implementing core modules with +:source:`pypy/module/` contains :ref:`mixed modules <mixed-modules>` + implementing core modules with both application and interpreter level code. Not all are finished and working. Use the ``--withmod-xxx`` or ``--allworkingmodules`` translation options. -:source:`pypy/objspace/` `object space`_ implementations +:source:`pypy/objspace/` :doc:`object space <objspace>` implementations -:source:`pypy/objspace/std/` the StdObjSpace_ implementing CPython's +:source:`pypy/objspace/std/` the :ref:`StdObjSpace <standard-object-space>` implementing CPython's objects and types :source:`pypy/tool/` various utilities and hacks used - from various places + from various places :source:`pypy/tool/algo/` general-purpose algorithmic and mathematic tools -:source:`pypy/tool/pytest/` support code for our `testing methods`_ - - -:source:`rpython/annotator/` `type inferencing code`_ for - `RPython`_ programs - -:source:`rpython/config/` handles the numerous options for RPython - - -:source:`rpython/flowspace/` the FlowObjSpace_ implementing - `abstract interpretation`_ - -:source:`rpython/rlib/` a `"standard library"`_ for RPython_ - programs - -:source:`rpython/rtyper/` the `RPython Typer`_ - -:source:`rpython/rtyper/lltypesystem/` the `low-level type system`_ for - C-like backends - -:source:`rpython/rtyper/ootypesystem/` the `object-oriented type system`_ - for OO backends - -:source:`rpython/memory/` the `garbage collector`_ construction - framework - -:source:`rpython/translator/` translation_ backends and support code - -:source:`rpython/translator/backendopt/` general optimizations that run before a - backend generates code - -:source:`rpython/translator/c/` the `GenC backend`_, producing C code - from an - RPython program (generally via the rtyper_) - -:source:`rpython/translator/cli/` the `CLI backend`_ for `.NET`_ - (Microsoft CLR or Mono_) - -:source:`pypy/goal/` our `main PyPy-translation scripts`_ - live here - -:source:`rpython/translator/jvm/` the Java backend - -:source:`rpython/translator/tool/` helper tools for translation - -:source:`dotviewer/` `graph viewer`_ +:source:`pypy/tool/pytest/` support code for our :ref:`testing methods <testing>` ``*/test/`` many directories have a test subdirectory - containing test - modules (see `Testing in PyPy`_) + containing test + modules (see :ref:`testing`) ``_cache/`` holds cache files from various purposes ======================================== ============================================ - -.. _`bytecode interpreter`: interpreter.html -.. _`Testing in PyPy`: coding-guide.html#testing-in-pypy -.. _`mixed modules`: coding-guide.html#mixed-modules -.. _`modules`: coding-guide.html#modules -.. _`basil`: http://people.cs.uchicago.edu/~jriehl/BasilTalk.pdf -.. _`object space`: objspace.html -.. _FlowObjSpace: objspace.html#the-flow-object-space -.. _`transparent proxies`: objspace-proxies.html#tproxy -.. _`Differences between PyPy and CPython`: cpython_differences.html -.. _`What PyPy can do for your objects`: objspace-proxies.html -.. _`Continulets and greenlets`: stackless.html -.. _StdObjSpace: objspace.html#the-standard-object-space -.. _`abstract interpretation`: http://en.wikipedia.org/wiki/Abstract_interpretation -.. _`rpython`: coding-guide.html#rpython -.. _`type inferencing code`: translation.html#the-annotation-pass -.. _`RPython Typer`: translation.html#rpython-typer -.. _`testing methods`: coding-guide.html#testing-in-pypy -.. _`translation`: translation.html -.. _`GenC backend`: translation.html#genc -.. _`CLI backend`: cli-backend.html -.. _`py.py`: getting-started-python.html#the-py.py-interpreter -.. _`translatorshell.py`: getting-started-dev.html#try-out-the-translator -.. _JIT: jit/index.html -.. _`JIT Generation in PyPy`: jit/index.html -.. _`just-in-time compiler generator`: jit/index.html -.. _rtyper: rtyper.html -.. _`low-level type system`: rtyper.html#low-level-type -.. _`object-oriented type system`: rtyper.html#oo-type -.. _`garbage collector`: garbage_collection.html -.. _`main PyPy-translation scripts`: getting-started-python.html#translating-the-pypy-python-interpreter -.. _`.NET`: http://www.microsoft.com/net/ -.. _Mono: http://www.mono-project.com/ -.. _`"standard library"`: rlib.html -.. _`graph viewer`: getting-started-dev.html#try-out-the-translator diff --git a/pypy/doc/discussion/VM-integration.rst b/pypy/doc/discussion/VM-integration.rst index bf7968cb58..a9ba9be6b9 100644 --- a/pypy/doc/discussion/VM-integration.rst +++ b/pypy/doc/discussion/VM-integration.rst @@ -1,4 +1,3 @@ -============================================== Integration of PyPy with host Virtual Machines ============================================== @@ -10,8 +9,9 @@ Terminology disclaimer: both PyPy and .NET have the concept of "wrapped" or "boxed" objects. To avoid confusion I will use "wrapping" on the PyPy side and "boxing" on the .NET side. + General idea -============ +------------ The goal is to find a way to efficiently integrate the PyPy interpreter with the hosting environment such as .NET. What we would @@ -29,8 +29,9 @@ like to do includes but it's not limited to: One possible solution is the "proxy" approach, in which we manually (un)wrap/(un)box all the objects when they cross the border. + Example -------- +~~~~~~~ :: @@ -43,8 +44,9 @@ In this case we need to take the intval field of W_IntObject, box it to .NET System.Int32, call foo using reflection, then unbox the return value and reconstruct a new (or reuse an existing one) W_IntObject. + The other approach ------------------- +~~~~~~~~~~~~~~~~~~ The general idea to solve handle this problem is to split the "stateful" and "behavioral" parts of wrapped objects, and use already @@ -57,7 +59,7 @@ reconstruct the pair. Split state and behaviour in the flowgraphs -=========================================== +------------------------------------------- The idea is to write a graph transformation that takes an usual ootyped flowgraph and split the classes and objects we want into a @@ -83,15 +85,16 @@ of its first member. identity (in which the two objects will have a different identity, of course). + Step 1: RPython source code ---------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~~ :: class W_IntObject: def __init__(self, intval): self.intval = intval - + def foo(self, x): return self.intval + x @@ -101,7 +104,7 @@ Step 1: RPython source code Step 2: RTyping ---------------- +~~~~~~~~~~~~~~~ Sometimes the following examples are not 100% accurate for the sake of simplicity (e.g: we directly list the type of methods instead of the @@ -149,8 +152,9 @@ Flowgraphs 3. return result } + Step 3: Transformation ----------------------- +~~~~~~~~~~~~~~~~~~~~~~ This step is done before the backend plays any role, but it's still driven by its need, because at this time we want a mapping that tell @@ -225,7 +229,7 @@ Flowgraphs Inheritance ------------ +~~~~~~~~~~~ Apply the transformation to a whole class (sub)hierarchy is a bit more complex. Basically we want to mimic the same hierarchy also on the @@ -259,5 +263,3 @@ This means that the low level type of all the ``W_Object`` subclasses will be ``W_Object_pair``, but it also means that we will need to insert the appropriate downcasts every time we want to access its fields. I'm not sure how much this can impact performances. - - diff --git a/pypy/doc/discussion/howtoimplementpickling.rst b/pypy/doc/discussion/howtoimplementpickling.rst index e5dc7288e9..2836cc8447 100644 --- a/pypy/doc/discussion/howtoimplementpickling.rst +++ b/pypy/doc/discussion/howtoimplementpickling.rst @@ -161,24 +161,24 @@ Examples are - map is a stateful implementation of iterating over a sequence of operations. It can be made non-recursive if the map operation creates its own frame to keep state. - + - __init__ looks trivial, but the semantics is that the return value of __init__ is supposed to be None, and CPy has a special check for this after the call. This might simply be ignored, but it is a simple example for a case that cannot be handled automatically. - + - things like operator.__add__ can theoretically generate a wild pattern of recursive calls while CPy tries to figure out if it is a numeric add or a sequence add, and other callbacks may occur when methods like __coerce__ get involved. This will never be solved for SLP, but might get a solution by the strategy outlined below. - + The second implementation took a radically different approach. Context switches were done by hijacking parts of the C stack, storing them away and replacing them by the stack fragment that the target needs. This is very powerful and allows to switch even in the context of foreign code. With a little risk, I was even able to add concurrency -to foreign Fortran code. +to foreign Fortran code. The above concept is called Hard (switching), the collaborative Soft (switching). Note that an improved version of Hard is still the building block @@ -244,11 +244,11 @@ we really need to restore before we can do the function call? - the argument decoding is done, already, and the fact that we could have done the function call shows, that no exception occurred. We can ignore the rest of this activation record and do the housekeeping. - + - the frame is prepared, and arguments are stored in it. The operation succeeded, and we have the frame. We can ignore exception handling and just do housekeeping by getting rid of references. - + - for executing the frame, we need a special function that executes frames. It is possible that we need different flavors due to contexts. SLP does this by using different registered functions which operate on a frame, depending @@ -327,7 +327,7 @@ track references which are known to be held by other objects. Optimizing that would do two things: The refcounting would become very efficient, since we would save some 80 % of it. The second part, which is relevant to the pickling problem is this: -By doing a proper analysis, we already would have lost references to +By doing a proper analysis, we already would have lost references to all the variables which we don't need to save any longer, because we know that they are held in, for instance, frames. diff --git a/pypy/doc/discussion/outline-external-ootype.rst b/pypy/doc/discussion/outline-external-ootype.rst index a01521df84..c295b18847 100644 --- a/pypy/doc/discussion/outline-external-ootype.rst +++ b/pypy/doc/discussion/outline-external-ootype.rst @@ -54,7 +54,7 @@ PythonNet; for JVM, there are JPype_ and JTool_, to be investigated: How to represent types ---------------------- -First, some definitions: +First, some definitions: - high-level types are the types used by the annotator (SomeInteger() & co.) diff --git a/pypy/doc/discussions.rst b/pypy/doc/discussions.rst index 716c16f62d..c77aa6dcb8 100644 --- a/pypy/doc/discussions.rst +++ b/pypy/doc/discussions.rst @@ -7,7 +7,7 @@ Help from domain experts would be welcome, since some of these documents probabl .. toctree:: - + discussion/finalizer-order.rst discussion/howtoimplementpickling.rst discussion/improve-rpython.rst diff --git a/pypy/doc/eventhistory.rst b/pypy/doc/eventhistory.rst index 5c42bba1a0..e53fdeff0e 100644 --- a/pypy/doc/eventhistory.rst +++ b/pypy/doc/eventhistory.rst @@ -1,37 +1,39 @@ ------------- More sprints ------------- +============ The PyPy project is a worldwide collaborative effort and its members are organizing sprints and presenting results at conferences all year round. **This page is no longer maintained!** See `our blog`_ - for upcoming events. + for upcoming events. + +.. _our blog: http://morepypy.blogspot.com/ -.. _`our blog`: http://morepypy.blogspot.com/ EuroPython PyPy sprint 6-9 July 2006 -================================================================== +------------------------------------------------------------------ Once again a PyPy sprint took place right after the EuroPython Conference from the *6th to the 9th of July*. Read more in the `EuroPython 2006 sprint report`_. -.. _`EuroPython 2006 sprint report`: http://codespeak.net/pypy/extradoc/sprintinfo/post-ep2006/report.txt +.. _EuroPython 2006 sprint report: http://codespeak.net/pypy/extradoc/sprintinfo/post-ep2006/report.txt + PyPy at XP 2006 and Agile 2006 -================================================================== +------------------------------------------------------------------ PyPy presented experience reports at the two main agile conferences this year, `XP 2006`_ and `Agile 2006`_. Both experience reports focus on aspects of the sprint-driven development method that is being used in PyPy. -.. _`XP 2006`: http://virtual.vtt.fi/virtual/xp2006/ -.. _`Agile 2006`: http://www.agile2006.org/ +.. _XP 2006: http://virtual.vtt.fi/virtual/xp2006/ +.. _Agile 2006: http://www.agile2006.org/ + Duesseldorf PyPy sprint 2-9 June 2006 -================================================================== +------------------------------------------------------------------ The next PyPy sprint will be held in the Computer Science department of Heinrich-Heine Universitaet Duesseldorf from the *2nd to the 9th of June*. @@ -41,11 +43,12 @@ release. Read more in `the sprint announcement`_, see who is planning to attend on the `people page`_. -.. _`the sprint announcement`: http://codespeak.net/pypy/extradoc/sprintinfo/ddorf2006/announce.html -.. _`people page`: http://codespeak.net/pypy/extradoc/sprintinfo/ddorf2006/people.html +.. _the sprint announcement: http://codespeak.net/pypy/extradoc/sprintinfo/ddorf2006/announce.html +.. _people page: http://codespeak.net/pypy/extradoc/sprintinfo/ddorf2006/people.html + PyPy sprint at Akihabara (Tokyo, Japan) -================================================================== +------------------------------------------------------------------ *April 23rd - 29th 2006.* This sprint was in Akihabara, Tokyo, Japan, our hosts was FSIJ (Free Software Initiative of Japan) and we aimed @@ -54,29 +57,32 @@ progress was also made on PyPy's ootypesystem for the more high level backends. For more details, read the last `sprint status`_ page and enjoy the pictures_. -.. _`sprint status`: https://bitbucket.org/pypy/extradoc/src/tip/sprintinfo/tokyo/tokyo-planning.txt -.. _`pictures`: http://www.flickr.com/photos/19046555@N00/sets/72057594116388174/ +.. _sprint status: https://bitbucket.org/pypy/extradoc/src/tip/sprintinfo/tokyo/tokyo-planning.txt +.. _pictures: http://www.flickr.com/photos/19046555@N00/sets/72057594116388174/ + PyPy at Python UK/ACCU Conference (United Kingdom) -=================================================================== +------------------------------------------------------------------- *April 19th - April 22nd 2006.* Several talks about PyPy were hold at this year's Python UK/ACCU conference. Read more at the `ACCU site`_. -.. _`ACCU site`: http://accu.org/ +.. _ACCU site: http://accu.org/ + PyPy at XPDay France 2006 in Paris March 23rd - March 24th 2006 -================================================================== +------------------------------------------------------------------ Logilab presented PyPy at the first french XP Day that it was sponsoring and which was held in Paris. There was over a hundred attendants. Interesting talks included Python as an agile language and Tools for continuous integration. - -.. _`french XP Day`: http://www.xpday.fr/ + +.. _french XP Day: http://www.xpday.fr/ + Logic Sprint at Louvain-la-Neuve University (Louvain-la-Neuve, Belgium) -======================================================================== +------------------------------------------------------------------------ *March 6th - March 10th 2006.* PyPy developers focusing on adding logic programming to PyPy will met with the team that developed the Oz @@ -87,8 +93,9 @@ Read the report_ and the original announcement_. .. _report: http://codespeak.net/pypy/extradoc/sprintinfo/louvain-la-neuve-2006/report.html .. _announcement: http://codespeak.net/pypy/extradoc/sprintinfo/louvain-la-neuve-2006/sprint-announcement.html + PyCon Sprint 2006 (Dallas, Texas, USA) -================================================================== +------------------------------------------------------------------ *Feb 27th - March 2nd 2006.* The Post-PyCon PyPy Sprint took place right after PyCon 2006. @@ -97,15 +104,15 @@ A report is coming up. Talks at PyCon 2006 (Dallas, Texas, USA) -=================================================================== +------------------------------------------------------------------- *Feb 24th - Feb 26th 2006.* PyPy developers spoke at PyCon 2006. -.. _`PyCon 2006`: http://us.pycon.org/TX2006/HomePage +.. _PyCon 2006: http://us.pycon.org/TX2006/HomePage PyPy at Solutions Linux in Paris January 31st - February 2nd 2006 -=================================================================== +------------------------------------------------------------------- PyPy developers from Logilab presented the intermediate results of the project during the Solutions Linux tradeshow in Paris. A lot of @@ -114,11 +121,11 @@ about the details. Many people discovered PyPy on this occasion and said they were interested in the outcome and would keep an eye on its progress. Read the `talk slides`_. -.. _`talk slides`: http://codespeak.net/pypy/extradoc/talk/solutions-linux-paris-2006.html +.. _talk slides: http://codespeak.net/pypy/extradoc/talk/solutions-linux-paris-2006.html PyPy Sprint in Palma De Mallorca 23rd - 29th January 2006 -=================================================================== +------------------------------------------------------------------- The Mallorca sprint that took place in Palma de Mallorca is over. Topics included progressing with the JIT work started in Göteborg @@ -128,154 +135,145 @@ improving our way to write glue code for C libraries. Read more in `the announcement`_, there is a `sprint report`_ for the first three days and `one for the rest of the sprint`_. +.. _the announcement: http://codespeak.net/pypy/extradoc/sprintinfo/mallorca/sprint-announcement.html +.. _sprint report: http://codespeak.net/pipermail/pypy-dev/2006q1/002746.html +.. _one for the rest of the sprint: http://codespeak.net/pipermail/pypy-dev/2006q1/002749.html -.. _`the announcement`: http://codespeak.net/pypy/extradoc/sprintinfo/mallorca/sprint-announcement.html -.. _`sprint report`: http://codespeak.net/pipermail/pypy-dev/2006q1/002746.html -.. _`one for the rest of the sprint`: http://codespeak.net/pipermail/pypy-dev/2006q1/002749.html Preliminary EU reports released -=============================== +------------------------------- After many hours of writing and typo-hunting we finally finished the -`reports for the EU`_. They contain most of the material found on our regular +:doc:`reports for the EU <index-report>`. They contain most of the material found on our regular documentation page but also a lot of new material not covered there. Note that all these documents are not approved by the European Union and therefore only preliminary. *(01/06/2006)* -.. _`reports for the EU`: index-report.html - -PyPy Sprint in Göteborg 7th - 11th December 2005 -================================================= +PyPy Sprint in Göteborg 7th - 11th December 2005 +------------------------------------------------ The Gothenburg sprint is over. It was a very productive sprint: work has -been started on a JIT prototype, we added support for __del__ in PyPy, +been started on a JIT prototype, we added support for __del__ in PyPy, the socket module had some progress, PyPy got faster and work was started to expose the internals of our parser and bytecode compiler to the user. Michael and Carl have written a `report about the first half`_ and `one about the second half`_ of the sprint. *(12/18/2005)* -.. _`report about the first half`: http://codespeak.net/pipermail/pypy-dev/2005q4/002656.html -.. _`one about the second half`: http://codespeak.net/pipermail/pypy-dev/2005q4/002660.html +.. _report about the first half: http://codespeak.net/pipermail/pypy-dev/2005q4/002656.html +.. _one about the second half: http://codespeak.net/pipermail/pypy-dev/2005q4/002660.html + PyPy release 0.8.0 -=================== +------------------ The third PyPy release is out, with an integrated and translatable compiler, speed progress, and now the possibility to translate our experimental "Thunk" object space (supporting lazy computed objects) with its features preserved. -See the `release 0.8 announcement`_ for further details about the release and -the `getting started`_ document for instructions about downloading it and -trying it out. There is also a short FAQ_. *(11/03/2005)* +See the :doc:`release 0.8 announcement <releases/0.8.0>` for further details about the release and +the :doc:`getting started <getting-started>` document for instructions about downloading it and +trying it out. There is also a short :doc:`FAQ <faq>`. *(11/03/2005)* -.. _`release 0.8 announcement`: release-0.8.0.html -PyPy Sprint in Paris 10th-16th October 2005 -======================================================== +PyPy Sprint in Paris 10th-16th October 2005 +------------------------------------------- The Paris sprint is over. We are all at home again and more or less exhausted. The sprint attracted 18 participants and took place in -`Logilab offices in Paris`_. We were happy to have five new +`Logilab offices in Paris`_. We were happy to have five new developers to the PyPy Community! The focus was on implementing `continuation-passing`_ style (stackless), making the translation process work for target languages with more powerful object systems and some tiny steps into the JIT_ direction. Michael and Carl have written -a `report about day one`_ and `one about day two and three`_. +a `report about day one`_ and `one about day two and three`_. Together with Armin they wrote one about `the rest of the sprint`_ on the way back. *(10/18/2005)* -.. _`Logilab offices in Paris`: http://codespeak.net/pypy/extradoc/sprintinfo/paris-2005-sprint.html +.. _Logilab offices in Paris: http://codespeak.net/pypy/extradoc/sprintinfo/paris-2005-sprint.html .. _JIT: http://en.wikipedia.org/wiki/Just-in-time_compilation -.. _`continuation-passing`: http://en.wikipedia.org/wiki/Continuation_passing_style -.. _`report about day one`: http://codespeak.net/pipermail/pypy-dev/2005q4/002510.html -.. _`one about day two and three`: http://codespeak.net/pipermail/pypy-dev/2005q4/002512.html -.. _`the rest of the sprint`: http://codespeak.net/pipermail/pypy-dev/2005q4/002514.html +.. _continuation-passing: http://en.wikipedia.org/wiki/Continuation_passing_style +.. _report about day one: http://codespeak.net/pipermail/pypy-dev/2005q4/002510.html +.. _one about day two and three: http://codespeak.net/pipermail/pypy-dev/2005q4/002512.html +.. _the rest of the sprint: http://codespeak.net/pipermail/pypy-dev/2005q4/002514.html + PyPy release 0.7.0 -=================== +------------------ The first implementation of Python in Python is now also the second implementation of Python in C :-) -See the `release announcement`_ for further details about the release and -the `getting started`_ document for instructions about downloading it and -trying it out. We also have the beginning of a FAQ_. *(08/28/2005)* +See the :doc:`release announcement <releases/0.7.0>` for further details about the release and +the :doc:`getting started <getting-started>` document for instructions about downloading it and +trying it out. We also have the beginning of a :doc:`FAQ <faq>`. *(08/28/2005)* -.. _`pypy-0.7.0`: -.. _`release announcement`: release-0.7.0.html -.. _`getting started`: getting-started.html -.. _FAQ: faq.html PyPy Sprint in Heidelberg 22nd-29th August 2005 -========================================================== - -The last `PyPy sprint`_ took place at the Heidelberg University -in Germany from 22nd August to 29th August (both days included). -Its main focus is translation of the whole PyPy interpreter -to a low level language and reaching 2.4.1 Python compliance. -The goal of the sprint is to release a first self-contained -PyPy-0.7 version. Carl has written a report about `day 1 - 3`_, -there are `some pictures`_ online and a `heidelberg summary report`_ -detailing some of the works that led to the successful release -of `pypy-0.7.0`_! - -.. _`heidelberg summary report`: http://codespeak.net/pypy/extradoc/sprintinfo/Heidelberg-report.html -.. _`PyPy sprint`: http://codespeak.net/pypy/extradoc/sprintinfo/Heidelberg-sprint.html -.. _`day 1 - 3`: http://codespeak.net/pipermail/pypy-dev/2005q3/002287.html -.. _`some pictures`: http://codespeak.net/~hpk/heidelberg-sprint/ - -PyPy Hildesheim2 finished: first self-contained PyPy run! -=========================================================== - -Up until 31st August we were in a PyPy sprint at `Trillke-Gut`_. -Carl has written a `report about day 1`_, Holger -about `day 2 and day 3`_ and Carl again about `day 4 and day 5`_, -On `day 6`_ Holger reports the `breakthrough`_: PyPy runs -on its own! Hurray_!. And Carl finally reports about the winding -down of `day 7`_ which saw us relaxing, discussing and generally -having a good time. You might want to look at the selected -`pictures from the sprint`_. - -.. _`report about day 1`: http://codespeak.net/pipermail/pypy-dev/2005q3/002217.html -.. _`day 2 and day 3`: http://codespeak.net/pipermail/pypy-dev/2005q3/002220.html -.. _`day 4 and day 5`: http://codespeak.net/pipermail/pypy-dev/2005q3/002234.html -.. _`day 6`: http://codespeak.net/pipermail/pypy-dev/2005q3/002239.html -.. _`day 7`: http://codespeak.net/pipermail/pypy-dev/2005q3/002245.html -.. _`breakthrough`: http://codespeak.net/~hpk/hildesheim2-sprint-www/hildesheim2-sprint-www-Thumbnails/36.jpg -.. _`hurray`: http://codespeak.net/~hpk/hildesheim2-sprint-www/hildesheim2-sprint-www-Pages/Image37.html -.. _`pictures from the sprint`: http://codespeak.net/~hpk/hildesheim2-sprint-www/ -.. _`Trillke-Gut`: http://www.trillke.net - -EuroPython 2005 sprints finished -====================================================== - -We had two sprints around EuroPython, one more internal core -developer one and a public one. Both sprints were quite -successful. Regarding the Pre-EuroPython sprint Michael Hudson -has posted summaries of `day 1`_, `day 2`_ and `day 3`_ on -the `pypy-dev`_ mailing list. The larger public sprint -has not been summarized yet but it went very well. We had -20 people initially attending to hear the tutorials and -work a bit. Later with around 13-14 people we made the -move to Python-2.4.1, integrated the parser, improved -the LLVM backends and type inference in general. -*(07/13/2005)* - -.. _`day 1`: http://codespeak.net/pipermail/pypy-dev/2005q2/002169.html -.. _`day 2`: http://codespeak.net/pipermail/pypy-dev/2005q2/002171.html -.. _`day 3`: http://codespeak.net/pipermail/pypy-dev/2005q2/002172.html -.. _`pypy-dev`: http://python.org/mailman/listinfo/pypy-dev - -.. _EuroPython: http://europython.org -.. _`translation`: translation.html -.. _`sprint announcement`: http://codespeak.net/pypy/extradoc/sprintinfo/EP2005-announcement.html -.. _`list of people coming`: http://codespeak.net/pypy/extradoc/sprintinfo/EP2005-people.html +----------------------------------------------- + +The last `PyPy sprint`_ took place at the Heidelberg University in Germany from +22nd August to 29th August (both days included). Its main focus is translation +of the whole PyPy interpreter to a low level language and reaching 2.4.1 Python +compliance. The goal of the sprint is to release a first self-contained +PyPy-0.7 version. Carl has written a report about `day 1 - 3`_, there are +`some pictures`_ online and a `heidelberg summary report`_ detailing some of +the works that led to the successful release of :doc:`pypy-0.7.0 <releases/0.7.0>`! + +.. _heidelberg summary report: http://codespeak.net/pypy/extradoc/sprintinfo/Heidelberg-report.html +.. _PyPy sprint: http://codespeak.net/pypy/extradoc/sprintinfo/Heidelberg-sprint.html +.. _day 1 - 3: http://codespeak.net/pipermail/pypy-dev/2005q3/002287.html +.. _some pictures: http://codespeak.net/~hpk/heidelberg-sprint/ + + +PyPy Hildesheim2 finished: first self-contained PyPy run! +--------------------------------------------------------- + +Up until 31st August we were in a PyPy sprint at `Trillke-Gut`_. Carl has +written a `report about day 1`_, Holger about `day 2 and day 3`_ and Carl again +about `day 4 and day 5`_, On `day 6`_ Holger reports the `breakthrough`_: PyPy +runs on its own! Hurray_!. And Carl finally reports about the winding down of +`day 7`_ which saw us relaxing, discussing and generally having a good time. +You might want to look at the selected `pictures from the sprint`_. + +.. _report about day 1: http://codespeak.net/pipermail/pypy-dev/2005q3/002217.html +.. _day 2 and day 3: http://codespeak.net/pipermail/pypy-dev/2005q3/002220.html +.. _day 4 and day 5: http://codespeak.net/pipermail/pypy-dev/2005q3/002234.html +.. _day 6: http://codespeak.net/pipermail/pypy-dev/2005q3/002239.html +.. _day 7: http://codespeak.net/pipermail/pypy-dev/2005q3/002245.html +.. _breakthrough: http://codespeak.net/~hpk/hildesheim2-sprint-www/hildesheim2-sprint-www-Thumbnails/36.jpg +.. _hurray: http://codespeak.net/~hpk/hildesheim2-sprint-www/hildesheim2-sprint-www-Pages/Image37.html +.. _pictures from the sprint: http://codespeak.net/~hpk/hildesheim2-sprint-www/ +.. _Trillke-Gut: http://www.trillke.net + + +EuroPython 2005 sprints finished +-------------------------------- + +We had two sprints around EuroPython, one more internal core developer one and +a public one. Both sprints were quite successful. Regarding the +Pre-EuroPython sprint Michael Hudson has posted summaries of `day 1`_, `day 2`_ +and `day 3`_ on the `pypy-dev`_ mailing list. The larger public sprint has not +been summarized yet but it went very well. We had 20 people initially +attending to hear the tutorials and work a bit. Later with around 13-14 people +we made the move to Python-2.4.1, integrated the parser, improved the LLVM +backends and type inference in general. +*(07/13/2005)* + +.. _day 1: http://codespeak.net/pipermail/pypy-dev/2005q2/002169.html +.. _day 2: http://codespeak.net/pipermail/pypy-dev/2005q2/002171.html +.. _day 3: http://codespeak.net/pipermail/pypy-dev/2005q2/002172.html +.. _pypy-dev: http://python.org/mailman/listinfo/pypy-dev + +.. _EuroPython: http://europython.org +.. _sprint announcement: http://codespeak.net/pypy/extradoc/sprintinfo/EP2005-announcement.html +.. _list of people coming: http://codespeak.net/pypy/extradoc/sprintinfo/EP2005-people.html + Duesseldorf PyPy sprint 2-9 June 2006 -================================================================== +------------------------------------------------------------------ The next PyPy sprint will be held in the Computer Science department of Heinrich-Heine Universitaet Duesseldorf from the *2nd to the 9th of June*. @@ -285,24 +283,24 @@ release. Read more in `the sprint announcement`_, see who is planning to attend on the `people page`_. -.. _`the sprint announcement`: http://codespeak.net/pypy/extradoc/sprintinfo/ddorf2006/announce.html -.. _`people page`: http://codespeak.net/pypy/extradoc/sprintinfo/ddorf2006/people.html +.. _the sprint announcement: http://codespeak.net/pypy/extradoc/sprintinfo/ddorf2006/announce.html +.. _people page: http://codespeak.net/pypy/extradoc/sprintinfo/ddorf2006/people.html PyPy at XP 2006 and Agile 2006 -================================================================== +------------------------------------------------------------------ PyPy will present experience reports at the two main agile conferences this year, `XP 2006`_ and `Agile 2006`_. Both experience reports focus on aspects of the sprint-driven development method that is being used in PyPy. -.. _`XP 2006`: http://virtual.vtt.fi/virtual/xp2006/ -.. _`Agile 2006`: http://www.agile2006.org/ +.. _XP 2006: http://virtual.vtt.fi/virtual/xp2006/ +.. _Agile 2006: http://www.agile2006.org/ EuroPython PyPy sprint 6-9 July 2006 -================================================================== +------------------------------------------------------------------ Once again a PyPy sprint will take place right after the EuroPython Conference. This year it will be from the *6th to the 9th of July*. @@ -310,6 +308,6 @@ Conference. This year it will be from the *6th to the 9th of July*. Read more in `EuroPython sprint announcement`_, see who is planning to attend on `the people page`_. There is also a page_ in the python wiki. -.. _`EuroPython sprint announcement`: https://bitbucket.org/pypy/extradoc/src/tip/sprintinfo/post-ep2006/announce.txt -.. _`the people page`: https://bitbucket.org/pypy/extradoc/src/tip/sprintinfo/post-ep2006/people.txt +.. _EuroPython sprint announcement: https://bitbucket.org/pypy/extradoc/src/tip/sprintinfo/post-ep2006/announce.txt +.. _the people page: https://bitbucket.org/pypy/extradoc/src/tip/sprintinfo/post-ep2006/people.txt .. _page: http://wiki.python.org/moin/EuroPython2006 diff --git a/pypy/doc/extending.rst b/pypy/doc/extending.rst index 33d0ef6542..b747e7f089 100644 --- a/pypy/doc/extending.rst +++ b/pypy/doc/extending.rst @@ -1,6 +1,5 @@ -=================================== Writing extension modules for pypy -=================================== +================================== This document tries to explain how to interface the PyPy python interpreter with any external library. @@ -9,28 +8,25 @@ Note: We try to describe state-of-the art, but it might fade out of date as this is the front on which things are changing in pypy rapidly. + Possibilities -============= +------------- Right now, there are three possibilities of providing third-party modules for the PyPy python interpreter (in order of usefulness): -* Write them in pure python and use ctypes, see ctypes_ - section +* Write them in pure python and use ctypes, see CTypes_ section * Write them in pure python and use direct libffi low-level bindings, See - \_ffi_ module description. + `\_ffi <LibFFI>`_ module description. -* Write them in RPython as mixedmodule_, using *rffi* as bindings. +* Write them in RPython as `mixedmodule <Mixed Modules>`_, using *rffi* as bindings. * Write them in C++ and bind them through Reflex_ -.. _ctypes: #CTypes -.. _\_ffi: #LibFFI -.. _mixedmodule: #Mixed Modules CTypes -====== +------ The ctypes module in PyPy is ready to use. It's goal is to be as-compatible-as-possible with the @@ -51,16 +47,18 @@ check/convert the types of the parameters. Moreover, even if most calls are optimized, some cannot and thus need to follow the slow path, not optimized by the JIT. -.. _`ctypes-configure`: ctypes-implementation.html#ctypes-configure -.. _`CPython ctypes`: http://docs.python.org/library/ctypes.html +.. _ctypes-configure: ctypes-implementation.html#ctypes-configure +.. _CPython ctypes: http://docs.python.org/library/ctypes.html + Pros ----- +~~~~ Stable, CPython-compatible API. Most calls are fast, optimized by JIT. + Cons ----- +~~~~ Problems with platform-dependency (although we partially solve those). Although the JIT optimizes ctypes calls, some overhead is still @@ -68,7 +66,7 @@ present. The slow-path is very slow. LibFFI -====== +------ Mostly in order to be able to write a ctypes module, we developed a very low-level libffi bindings called ``_ffi``. (libffi is a C-level library for dynamic calling, @@ -76,21 +74,24 @@ which is used by CPython ctypes). This library provides stable and usable API, although it's API is a very low-level one. It does not contain any magic. It is also optimized by the JIT, but has much less overhead than ctypes. + Pros ----- +~~~~ It Works. Probably more suitable for a delicate code where ctypes magic goes in a way. All calls are optimized by the JIT, there is no slow path as in ctypes. + Cons ----- +~~~~ It combines disadvantages of using ctypes with disadvantages of using mixed modules. CPython-incompatible API, very rough and low-level. + Mixed Modules -============= +------------- This is the most advanced and powerful way of writing extension modules. It has some serious disadvantages: @@ -106,18 +107,14 @@ It has some serious disadvantages: C and RPython needs to be managed by hand (each object that goes into the C level must be explicitly freed). -Some documentation is available `here`_ - -.. _`here`: rffi.html +Some documentation is available :doc:`here <rpython:rffi>` -XXX we should provide detailed docs about lltype and rffi, especially if we - want people to follow that way. Reflex -====== +------ This method is still experimental and is being exercised on a branch, -`reflex-support`_, which adds the `cppyy`_ module. +reflex-support, which adds the :doc:`cppyy <cppyy>` module. The method works by using the `Reflex package`_ to provide reflection information of the C++ code, which is then used to automatically generate bindings at runtime. @@ -128,15 +125,13 @@ runtime structures, created when a script or module loads. However, if the backend itself is capable of dynamic behavior, it is a much better functional match to python, allowing tighter integration and more natural language mappings. -Full details are `available here`_. +Full details are available :doc:`here <cppyy>`. + +.. _Reflex package: http://root.cern.ch/drupal/content/reflex -.. _`cppyy`: cppyy.html -.. _`reflex-support`: cppyy.html -.. _`Reflex package`: http://root.cern.ch/drupal/content/reflex -.. _`available here`: cppyy.html Pros ----- +~~~~ The cppyy module is written in RPython, which makes it possible to keep the code execution visible to the JIT all the way to the actual point of call into @@ -153,11 +148,12 @@ This will affect the backend only; the python-side interface is expected to remain the same, except that cling adds a lot of dynamic behavior to C++, enabling further language integration. -.. _`cling`: http://root.cern.ch/drupal/content/cling -.. _`llvm`: http://llvm.org/ +.. _cling: http://root.cern.ch/drupal/content/cling +.. _llvm: http://llvm.org/ + Cons ----- +~~~~ C++ is a large language, and cppyy is not yet feature-complete. Still, the experience gained in developing the equivalent bindings for CPython diff --git a/pypy/doc/extradoc.rst b/pypy/doc/extradoc.rst index b74afa4042..a4e7c64c95 100644 --- a/pypy/doc/extradoc.rst +++ b/pypy/doc/extradoc.rst @@ -1,13 +1,11 @@ -================================================= -Papers, talks and related projects -================================================= +Papers, talks and related projects +================================== Papers ----------------------------------- +------ *Articles about PyPy published so far, most recent first:* (bibtex_ file) - * `Runtime Feedback in a Meta-Tracing JIT for Efficient Dynamic Languages`_, C.F. Bolz, A. Cuni, M. Fijalkowski, M. Leuschel, S. Pedroni, A. Rigo @@ -71,36 +69,36 @@ Papers .. _bibtex: https://bitbucket.org/pypy/extradoc/raw/tip/talk/bibtex.bib -.. _`Runtime Feedback in a Meta-Tracing JIT for Efficient Dynamic Languages`: https://bitbucket.org/pypy/extradoc/raw/extradoc/talk/icooolps2011/jit-hints.pdf -.. _`Allocation Removal by Partial Evaluation in a Tracing JIT`: http://codespeak.net/svn/pypy/extradoc/talk/pepm2011/bolz-allocation-removal.pdf -.. _`Towards a Jitting VM for Prolog Execution`: http://www.stups.uni-duesseldorf.de/publications/bolz-prolog-jit.pdf -.. _`High performance implementation of Python for CLI/.NET with JIT compiler generation for dynamic languages`: http://buildbot.pypy.org/misc/antocuni-thesis.pdf -.. _`How to *not* write Virtual Machines for Dynamic Languages`: https://bitbucket.org/pypy/extradoc/raw/tip/talk/dyla2007/dyla.pdf -.. _`Tracing the Meta-Level: PyPy's Tracing JIT Compiler`: https://bitbucket.org/pypy/extradoc/raw/tip/talk/icooolps2009/bolz-tracing-jit.pdf -.. _`Faster than C#: Efficient Implementation of Dynamic Languages on .NET`: https://bitbucket.org/pypy/extradoc/raw/tip/talk/icooolps2009-dotnet/cli-jit.pdf -.. _`Automatic JIT Compiler Generation with Runtime Partial Evaluation`: http://www.stups.uni-duesseldorf.de/thesis/final-master.pdf -.. _`RPython: A Step towards Reconciling Dynamically and Statically Typed OO Languages`: http://www.disi.unige.it/person/AnconaD/papers/Recent_abstracts.html#AACM-DLS07 -.. _`EU Reports`: index-report.html -.. _`Hardware Transactional Memory Support for Lightweight Dynamic Language Evolution`: http://sabi.net/nriley/pubs/dls6-riley.pdf -.. _`PyGirl: Generating Whole-System VMs from High-Level Prototypes using PyPy`: http://scg.unibe.ch/archive/papers/Brun09cPyGirl.pdf -.. _`Representation-Based Just-in-Time Specialization and the Psyco Prototype for Python`: http://psyco.sourceforge.net/psyco-pepm-a.ps.gz -.. _`Back to the Future in One Week -- Implementing a Smalltalk VM in PyPy`: http://dx.doi.org/10.1007/978-3-540-89275-5_7 -.. _`Automatic generation of JIT compilers for dynamic languages in .NET`: https://bitbucket.org/pypy/extradoc/raw/tip/talk/ecoop2009/main.pdf -.. _`Core Object Optimization Results`: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D06.1_Core_Optimizations-2007-04-30.pdf -.. _`Compiling Dynamic Language Implementations`: http://codespeak.net/pypy/extradoc/eu-report/D05.1_Publish_on_translating_a_very-high-level_description.pdf - - -Talks and Presentations ----------------------------------- +.. _Runtime Feedback in a Meta-Tracing JIT for Efficient Dynamic Languages: https://bitbucket.org/pypy/extradoc/raw/extradoc/talk/icooolps2011/jit-hints.pdf +.. _Allocation Removal by Partial Evaluation in a Tracing JIT: http://codespeak.net/svn/pypy/extradoc/talk/pepm2011/bolz-allocation-removal.pdf +.. _Towards a Jitting VM for Prolog Execution: http://www.stups.uni-duesseldorf.de/publications/bolz-prolog-jit.pdf +.. _High performance implementation of Python for CLI/.NET with JIT compiler generation for dynamic languages: http://buildbot.pypy.org/misc/antocuni-thesis.pdf +.. _How to *not* write Virtual Machines for Dynamic Languages: https://bitbucket.org/pypy/extradoc/raw/tip/talk/dyla2007/dyla.pdf +.. _Tracing the Meta-Level\: PyPy's Tracing JIT Compiler: https://bitbucket.org/pypy/extradoc/raw/tip/talk/icooolps2009/bolz-tracing-jit.pdf +.. _Faster than C#\: Efficient Implementation of Dynamic Languages on .NET: https://bitbucket.org/pypy/extradoc/raw/tip/talk/icooolps2009-dotnet/cli-jit.pdf +.. _Automatic JIT Compiler Generation with Runtime Partial Evaluation: http://www.stups.uni-duesseldorf.de/thesis/final-master.pdf +.. _RPython\: A Step towards Reconciling Dynamically and Statically Typed OO Languages: http://www.disi.unige.it/person/AnconaD/papers/Recent_abstracts.html#AACM-DLS07 +.. _EU Reports: index-report.html +.. _Hardware Transactional Memory Support for Lightweight Dynamic Language Evolution: http://sabi.net/nriley/pubs/dls6-riley.pdf +.. _PyGirl\: Generating Whole-System VMs from High-Level Prototypes using PyPy: http://scg.unibe.ch/archive/papers/Brun09cPyGirl.pdf +.. _Representation-Based Just-in-Time Specialization and the Psyco Prototype for Python: http://psyco.sourceforge.net/psyco-pepm-a.ps.gz +.. _Back to the Future in One Week -- Implementing a Smalltalk VM in PyPy: http://dx.doi.org/10.1007/978-3-540-89275-5_7 +.. _Automatic generation of JIT compilers for dynamic languages in .NET: https://bitbucket.org/pypy/extradoc/raw/tip/talk/ecoop2009/main.pdf +.. _Core Object Optimization Results: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D06.1_Core_Optimizations-2007-04-30.pdf +.. _Compiling Dynamic Language Implementations: http://codespeak.net/pypy/extradoc/eu-report/D05.1_Publish_on_translating_a_very-high-level_description.pdf + + +Talks and Presentations +----------------------- Talks in 2010 -+++++++++++++ +~~~~~~~~~~~~~ * `PyCon 2010`_. Talks in 2009 -+++++++++++++ +~~~~~~~~~~~~~ * `RuPy 2009`_. @@ -115,7 +113,7 @@ Talks in 2009 Talks in 2008 -+++++++++++++ +~~~~~~~~~~~~~ * Talk `at PyCon Poland 08`_. In Polish. @@ -143,7 +141,7 @@ Talks in 2008 Talks in 2007 -+++++++++++++ +~~~~~~~~~~~~~ * Our "road show" tour of the United States: presentations `at IBM`__ and `at Google`__. @@ -180,7 +178,7 @@ Talks in 2007 Talks in 2006 -+++++++++++++ +~~~~~~~~~~~~~ * `Warsaw 2006`_. @@ -188,21 +186,21 @@ Talks in 2006 * `PyPy's VM Approach`_ talk, given by Armin Rigo at the Dynamic Languages Symposium at OOPSLA'06 (Portland OR), and by Samuele Pedroni at Intel - Hillsboro (OR) (October). The talk presents the paper - `PyPy's approach to virtual machine construction`_ accepted for + Hillsboro (OR) (October). The talk presents the paper + `PyPy's approach to virtual machine construction`_ accepted for the symposium. * `PyPy Status`_ talk, given by Samuele Pedroni at the Vancouner - Python Workshop 2006 (August). + Python Workshop 2006 (August). -* `Trouble in Paradise`_: the Open Source Project PyPy, +* `Trouble in Paradise`_: the Open Source Project PyPy, EU-funding and Agile Practices talk, by Bea During at Agile 2006 (experience report). * `Sprint Driven Development`_, Agile Methodologies in a Distributed Open Source Project (PyPy) talk, by Bea During at XP 2006 (experience report). - + * `Kill -1`_: process refactoring in the PyPy project talk, by Bea During at the Agile track/Europython 2006. @@ -216,19 +214,19 @@ Talks in 2006 stating the status of the project. * Very similar to the EuroPython intro talk (but somewhat older) is the - `PyPy intro`_ talk, given by Michael Hudson at ACCU 2006 (April) + `PyPy intro`_ talk, given by Michael Hudson at ACCU 2006 (April) * `PyPy development method`_ talk, given by Bea During and - Holger Krekel at Pycon2006 + Holger Krekel at Pycon2006 + Talks in 2005 -+++++++++++++ +~~~~~~~~~~~~~ +* `PyPy - the new Python implementation on the block`_, + given by Carl Friedrich Bolz and Holger Krekel at the + 22nd Chaos Communication Conference in Berlin, Dec. 2005. -* `PyPy - the new Python implementation on the block`_, - given by Carl Friedrich Bolz and Holger Krekel at the - 22nd Chaos Communication Conference in Berlin, Dec. 2005. - * `Open Source, EU-Funding and Agile Methods`_, given by Holger Krekel and Bea During at the 22nd Chaos Communication Conference in Berlin, Dec. 2005 @@ -238,85 +236,88 @@ Talks in 2005 * `PyCon 2005`_ animated slices, mostly reporting on the translator status. -* `py lib slides`_ from the py lib talk at PyCon 2005 - (py is used as a support/testing library for PyPy). +* `py lib slides`_ from the py lib talk at PyCon 2005 + (py is used as a support/testing library for PyPy). + Talks in 2004 -+++++++++++++ +~~~~~~~~~~~~~ * `EU funding for FOSS`_ talk on Chaos Communication - Conference in Berlin, Dec 2004. + Conference in Berlin, Dec 2004. + Talks in 2003 -+++++++++++++ +~~~~~~~~~~~~~ -* oscon2003-paper_ an early paper presented at Oscon 2003 describing - what the PyPy project is about and why you should care. +* oscon2003-paper_ an early paper presented at Oscon 2003 describing + what the PyPy project is about and why you should care. * `Architecture introduction slides`_ a mostly up-to-date - introduction for the Amsterdam PyPy-Sprint Dec 2003. - -.. _`PyCon 2010`: http://morepypy.blogspot.com/2010/02/pycon-2010-report.html -.. _`RuPy 2009`: http://morepypy.blogspot.com/2009/11/pypy-on-rupy-2009.html -.. _`PyPy 3000`: http://codespeak.net/pypy/extradoc/talk/ep2006/pypy3000.txt -.. _`What can PyPy do for you`: http://codespeak.net/pypy/extradoc/talk/ep2006/usecases-slides.html -.. _`PyPy introduction at EuroPython 2006`: http://codespeak.net/pypy/extradoc/talk/ep2006/intro.pdf -.. _`PyPy - the new Python implementation on the block`: http://codespeak.net/pypy/extradoc/talk/22c3/hpk-tech.html -.. _`PyPy development method`: http://codespeak.net/pypy/extradoc/talk/pycon2006/method_talk.html -.. _`PyPy intro`: http://codespeak.net/pypy/extradoc/talk/accu2006/accu-2006.pdf + introduction for the Amsterdam PyPy-Sprint Dec 2003. + + +.. _PyCon 2010: http://morepypy.blogspot.com/2010/02/pycon-2010-report.html +.. _RuPy 2009: http://morepypy.blogspot.com/2009/11/pypy-on-rupy-2009.html +.. _PyPy 3000: http://codespeak.net/pypy/extradoc/talk/ep2006/pypy3000.txt +.. _What can PyPy do for you: http://codespeak.net/pypy/extradoc/talk/ep2006/usecases-slides.html +.. _PyPy introduction at EuroPython 2006: http://codespeak.net/pypy/extradoc/talk/ep2006/intro.pdf +.. _PyPy - the new Python implementation on the block: http://codespeak.net/pypy/extradoc/talk/22c3/hpk-tech.html +.. _PyPy development method: http://codespeak.net/pypy/extradoc/talk/pycon2006/method_talk.html +.. _PyPy intro: http://codespeak.net/pypy/extradoc/talk/accu2006/accu-2006.pdf .. _oscon2003-paper: http://codespeak.net/pypy/extradoc/talk/oscon2003-paper.html -.. _`Architecture introduction slides`: http://codespeak.net/pypy/extradoc/talk/amsterdam-sprint-intro.pdf -.. _`EU funding for FOSS`: http://codespeak.net/pypy/extradoc/talk/2004-21C3-pypy-EU-hpk.pdf -.. _`py lib slides`: http://codespeak.net/pypy/extradoc/talk/2005-pycon-py.pdf -.. _`PyCon 2005`: http://codespeak.net/pypy/extradoc/talk/pypy-talk-pycon2005/README.html -.. _`Trouble in Paradise`: http://codespeak.net/pypy/extradoc/talk/agile2006/during-oss-sprints_talk.pdf -.. _`Sprint Driven Development`: http://codespeak.net/pypy/extradoc/talk/xp2006/during-xp2006-sprints.pdf -.. _`Kill -1`: http://codespeak.net/pypy/extradoc/talk/ep2006/kill_1_agiletalk.pdf -.. _`Open Source, EU-Funding and Agile Methods`: http://codespeak.net/pypy/extradoc/talk/22c3/agility.pdf -.. _`PyPy Status`: http://codespeak.net/pypy/extradoc/talk/vancouver/talk.html -.. _`Sprinting the PyPy way`: https://bitbucket.org/pypy/extradoc/raw/tip/talk/ep2005/pypy_sprinttalk_ep2005bd.pdf -.. _`PyPy's VM Approach`: http://codespeak.net/pypy/extradoc/talk/dls2006/talk.html -.. _`PyPy's approach to virtual machine construction`: https://bitbucket.org/pypy/extradoc/raw/tip/talk/dls2006/pypy-vm-construction.pdf -.. _`EuroPython talks 2009`: https://bitbucket.org/pypy/extradoc/raw/tip/talk/ep2009/ -.. _`PyCon talks 2009`: https://bitbucket.org/pypy/extradoc/raw/tip/talk/pycon2009/ -.. _`Wroclaw (Poland) presentation`: https://bitbucket.org/pypy/extradoc/raw/tip/talk/wroclaw2009/talk.pdf -.. _`PyPy talk at OpenBossa 09`: http://morepypy.blogspot.com/2009/03/pypy-talk-at-openbossa-09.html -.. _`at SFI 08`: https://bitbucket.org/pypy/extradoc/raw/tip/talk/sfi2008/ -.. _`at PyCon Poland 08`: https://bitbucket.org/pypy/extradoc/raw/tip/talk/pyconpl-2008/talk.pdf -.. _`The PyPy Project and You`: https://bitbucket.org/pypy/extradoc/raw/tip/talk/osdc2008/osdc08.pdf -.. _`EuroPython talks 2008`: https://bitbucket.org/pypy/extradoc/raw/tip/talk/ep2008/ -.. _`Maemo summit`: http://morepypy.blogspot.com/2008/09/pypypython-at-maemo-summit.html -.. _`PyCon UK 2008 - JIT`: https://bitbucket.org/pypy/extradoc/raw/tip/talk/pycon-uk-2008/jit/pypy-vm.pdf -.. _`PyCon UK 2008 - Status`: https://bitbucket.org/pypy/extradoc/raw/tip/talk/pycon-uk-2008/status/status.pdf -.. _`PyCon Italy 2008`: https://bitbucket.org/pypy/extradoc/raw/tip/talk/pycon-italy-2008/pypy-vm.pdf -.. _`RuPy 2008`: https://bitbucket.org/pypy/extradoc/raw/tip/talk/rupy2008/ -.. _`RuPy 2007`: https://bitbucket.org/pypy/extradoc/raw/tip/talk/rupy2007/ -.. _`PyCon 2008`: https://bitbucket.org/pypy/extradoc/raw/tip/talk/pycon2008/ -.. _`ESUG 2007`: https://bitbucket.org/pypy/extradoc/raw/tip/talk/esug2007/ -.. _`Bern (Switzerland) 2007`: https://bitbucket.org/pypy/extradoc/raw/tip/talk/bern2007/ -.. _`PyCon UK 2007`: https://bitbucket.org/pypy/extradoc/raw/tip/talk/pyconuk07/ +.. _Architecture introduction slides: http://codespeak.net/pypy/extradoc/talk/amsterdam-sprint-intro.pdf +.. _EU funding for FOSS: http://codespeak.net/pypy/extradoc/talk/2004-21C3-pypy-EU-hpk.pdf +.. _py lib slides: http://codespeak.net/pypy/extradoc/talk/2005-pycon-py.pdf +.. _PyCon 2005: http://codespeak.net/pypy/extradoc/talk/pypy-talk-pycon2005/README.html +.. _Trouble in Paradise: http://codespeak.net/pypy/extradoc/talk/agile2006/during-oss-sprints_talk.pdf +.. _Sprint Driven Development: http://codespeak.net/pypy/extradoc/talk/xp2006/during-xp2006-sprints.pdf +.. _Kill -1: http://codespeak.net/pypy/extradoc/talk/ep2006/kill_1_agiletalk.pdf +.. _Open Source, EU-Funding and Agile Methods: http://codespeak.net/pypy/extradoc/talk/22c3/agility.pdf +.. _PyPy Status: http://codespeak.net/pypy/extradoc/talk/vancouver/talk.html +.. _Sprinting the PyPy way: https://bitbucket.org/pypy/extradoc/raw/tip/talk/ep2005/pypy_sprinttalk_ep2005bd.pdf +.. _PyPy's VM Approach: http://codespeak.net/pypy/extradoc/talk/dls2006/talk.html +.. _PyPy's approach to virtual machine construction: https://bitbucket.org/pypy/extradoc/raw/tip/talk/dls2006/pypy-vm-construction.pdf +.. _EuroPython talks 2009: https://bitbucket.org/pypy/extradoc/raw/tip/talk/ep2009/ +.. _PyCon talks 2009: https://bitbucket.org/pypy/extradoc/raw/tip/talk/pycon2009/ +.. _Wroclaw (Poland) presentation: https://bitbucket.org/pypy/extradoc/raw/tip/talk/wroclaw2009/talk.pdf +.. _PyPy talk at OpenBossa 09: http://morepypy.blogspot.com/2009/03/pypy-talk-at-openbossa-09.html +.. _at SFI 08: https://bitbucket.org/pypy/extradoc/raw/tip/talk/sfi2008/ +.. _at PyCon Poland 08: https://bitbucket.org/pypy/extradoc/raw/tip/talk/pyconpl-2008/talk.pdf +.. _The PyPy Project and You: https://bitbucket.org/pypy/extradoc/raw/tip/talk/osdc2008/osdc08.pdf +.. _EuroPython talks 2008: https://bitbucket.org/pypy/extradoc/raw/tip/talk/ep2008/ +.. _Maemo summit: http://morepypy.blogspot.com/2008/09/pypypython-at-maemo-summit.html +.. _PyCon UK 2008 - JIT: https://bitbucket.org/pypy/extradoc/raw/tip/talk/pycon-uk-2008/jit/pypy-vm.pdf +.. _PyCon UK 2008 - Status: https://bitbucket.org/pypy/extradoc/raw/tip/talk/pycon-uk-2008/status/status.pdf +.. _PyCon Italy 2008: https://bitbucket.org/pypy/extradoc/raw/tip/talk/pycon-italy-2008/pypy-vm.pdf +.. _RuPy 2008: https://bitbucket.org/pypy/extradoc/raw/tip/talk/rupy2008/ +.. _RuPy 2007: https://bitbucket.org/pypy/extradoc/raw/tip/talk/rupy2007/ +.. _PyCon 2008: https://bitbucket.org/pypy/extradoc/raw/tip/talk/pycon2008/ +.. _ESUG 2007: https://bitbucket.org/pypy/extradoc/raw/tip/talk/esug2007/ +.. _Bern (Switzerland) 2007: https://bitbucket.org/pypy/extradoc/raw/tip/talk/bern2007/ +.. _PyCon UK 2007: https://bitbucket.org/pypy/extradoc/raw/tip/talk/pyconuk07/ .. _Dresden: https://bitbucket.org/pypy/extradoc/raw/tip/talk/dresden/ -.. _`EuroPython 2007`: https://bitbucket.org/pypy/extradoc/raw/tip/talk/ep2007/ -.. _`Bad Honnef 2007`: https://bitbucket.org/pypy/extradoc/raw/tip/talk/badhonnef2007/talk.pdf -.. _`Dzug talk`: https://bitbucket.org/pypy/extradoc/raw/tip/talk/dzug2007/dzug2007.txt -.. _`PyCon 2007`: https://bitbucket.org/pypy/extradoc/raw/tip/talk/pycon2007/ -.. _`PyCon - Uno 2007`: https://bitbucket.org/pypy/extradoc/raw/tip/talk/pycon-uno2007/pycon07.pdf -.. _`Warsaw 2007`: https://bitbucket.org/pypy/extradoc/raw/tip/talk/warsaw2007/ -.. _`Warsaw 2006`: https://bitbucket.org/pypy/extradoc/raw/tip/talk/warsaw2006/ -.. _`Tokyo 2006`: https://bitbucket.org/pypy/extradoc/raw/tip/talk/tokyo/ +.. _EuroPython 2007: https://bitbucket.org/pypy/extradoc/raw/tip/talk/ep2007/ +.. _Bad Honnef 2007: https://bitbucket.org/pypy/extradoc/raw/tip/talk/badhonnef2007/talk.pdf +.. _Dzug talk: https://bitbucket.org/pypy/extradoc/raw/tip/talk/dzug2007/dzug2007.txt +.. _PyCon 2007: https://bitbucket.org/pypy/extradoc/raw/tip/talk/pycon2007/ +.. _PyCon - Uno 2007: https://bitbucket.org/pypy/extradoc/raw/tip/talk/pycon-uno2007/pycon07.pdf +.. _Warsaw 2007: https://bitbucket.org/pypy/extradoc/raw/tip/talk/warsaw2007/ +.. _Warsaw 2006: https://bitbucket.org/pypy/extradoc/raw/tip/talk/warsaw2006/ +.. _Tokyo 2006: https://bitbucket.org/pypy/extradoc/raw/tip/talk/tokyo/ -Related projects ----------------------------------- +Related projects +---------------- * TraceMonkey_ is using a tracing JIT, similar to the tracing JITs generated by our (in-progress) JIT generator. * Dynamo_ showcased `transparent dynamic optimization`_ - generating an optimized version of a binary program at runtime. + generating an optimized version of a binary program at runtime. * Tailoring Dynamo_ to interpreter implementations and challenges - - Gregory Sullivan et. al., + Gregory Sullivan et. al., `Dynamic Native Optimization of Native Interpreters`_. IVME 03. 2003. * Stackless_ is a recursion-free version of Python. @@ -326,20 +327,20 @@ Related projects * JikesRVM_ a research dynamic optimizing Java VM written in Java. * `Squeak`_ is a Smalltalk-80 implementation written in - Smalltalk, being used in `Croquet`_, an experimental - distributed multi-user/multi-programmer virtual world. + Smalltalk, being used in `Croquet`_, an experimental + distributed multi-user/multi-programmer virtual world. + +* `LLVM`_ the low level virtual machine project. -* `LLVM`_ the low level virtual machine project. +* `CLR under the hood`_ (powerpoint, works with open office) gives + a good introduction to the underlying models of Microsoft's Common + Language Runtime, the Intermediate Language, JIT and GC issues. -* `CLR under the hood`_ (powerpoint, works with open office) gives - a good introduction to the underlying models of Microsoft's Common - Language Runtime, the Intermediate Language, JIT and GC issues. - * spyweb translates Python programs to Scheme. (site unavailable) * Jython_ is a Python implementation in Java. -* IronPython_ a new Python implementation compiling Python into +* IronPython_ a new Python implementation compiling Python into Microsoft's Common Language Runtime (CLR) Intermediate Language (IL). * Tunes_ is not entirely unrelated. The web site changed a lot, but a @@ -347,20 +348,19 @@ Related projects through it is a lot of fun. .. _TraceMonkey: https://wiki.mozilla.org/JavaScript:TraceMonkey -.. _`CLR under the hood`: http://download.microsoft.com/download/2/4/d/24dfac0e-fec7-4252-91b9-fb2310603f14/CLRUnderTheHood.BradA.ppt -.. _Stackless: http://stackless.com +.. _CLR under the hood: http://download.microsoft.com/download/2/4/d/24dfac0e-fec7-4252-91b9-fb2310603f14/CLRUnderTheHood.BradA.ppt +.. _Stackless: http://stackless.com .. _Psyco: http://psyco.sourceforge.net .. _Jython: http://www.jython.org -.. _`Squeak`: http://www.squeak.org/ -.. _`Croquet`: http://www.opencroquet.org/ -.. _`transparent dynamic optimization`: http://www.hpl.hp.com/techreports/1999/HPL-1999-77.pdf +.. _Squeak: http://www.squeak.org/ +.. _Croquet: http://www.opencroquet.org/ +.. _transparent dynamic optimization: http://www.hpl.hp.com/techreports/1999/HPL-1999-77.pdf .. _Dynamo: http://www.hpl.hp.com/techreports/1999/HPL-1999-78.pdf -.. _testdesign: coding-guide.html#test-design .. _feasible: http://codespeak.net/pipermail/pypy-dev/2004q2/001289.html .. _rock: http://codespeak.net/pipermail/pypy-dev/2004q1/001255.html .. _LLVM: http://llvm.org/ .. _IronPython: http://ironpython.codeplex.com/ -.. _`Dynamic Native Optimization of Native Interpreters`: http://people.csail.mit.edu/gregs/dynamorio.html +.. _Dynamic Native Optimization of Native Interpreters: http://people.csail.mit.edu/gregs/dynamorio.html .. _JikesRVM: http://jikesrvm.org/ .. _Tunes: http://tunes.org -.. _`old Tunes Wiki`: http://buildbot.pypy.org/misc/cliki.tunes.org/ +.. _old Tunes Wiki: http://buildbot.pypy.org/misc/cliki.tunes.org/ diff --git a/pypy/doc/faq.rst b/pypy/doc/faq.rst index e9d78f7fb3..cf63e42e90 100644 --- a/pypy/doc/faq.rst +++ b/pypy/doc/faq.rst @@ -1,10 +1,8 @@ -========================== Frequently Asked Questions ========================== .. contents:: -------------- What is PyPy? ------------- @@ -13,14 +11,9 @@ toolchain. PyPy tries to find new answers about ease of creation, flexibility, maintainability and speed trade-offs for language implementations. -For further details see our `goal and architecture document`_ . - -.. _`goal and architecture document`: architecture.html +For further details see our :doc:`goal and architecture document <architecture>`. -.. _`drop in replacement`: - ------------------------------------------- Is PyPy a drop in replacement for CPython? ------------------------------------------ @@ -37,7 +30,7 @@ extension modules there is a good chance that it will work with PyPy. We list the differences we know about in `cpython differences`_. --------------------------------------------- + Do CPython Extension modules work with PyPy? -------------------------------------------- @@ -52,17 +45,15 @@ JIT can see. We fully support ctypes-based extensions. But for best performance, we recommend that you use the cffi_ module to interface with C code. -For information on which third party extensions work (or do not work) +For information on which third party extensions work (or do not work) with PyPy see the `compatibility wiki`_. - -.. _`extension modules`: cpython_differences.html#extension-modules -.. _`cpython differences`: cpython_differences.html -.. _`compatibility wiki`: -.. https://bitbucket.org/pypy/compatibility/wiki/Home +.. _extension modules: cpython_differences.html#extension-modules +.. _cpython differences: cpython_differences.html +.. _compatibility wiki: https://bitbucket.org/pypy/compatibility/wiki/Home .. _cffi: http://cffi.readthedocs.org/ ---------------------------------- + On which platforms does PyPy run? --------------------------------- @@ -74,41 +65,39 @@ At the moment you need CPython 2.5 - 2.7 for the translation process. PyPy's JIT requires an x86 or x86_64 CPU. (There has also been good progress on getting the JIT working for ARMv7.) ------------------------------------------------- + Which Python version (2.x?) does PyPy implement? ------------------------------------------------ PyPy currently aims to be fully compatible with Python 2.7. That means that it contains the standard library of Python 2.7 and that it supports 2.7 -features (such as set comprehensions). +features (such as set comprehensions). + .. _threading: -------------------------------------------------- Does PyPy have a GIL? Why? ------------------------------------------------- Yes, PyPy has a GIL. Removing the GIL is very hard. The first problem is that our garbage collectors are not re-entrant. ------------------------------------------- + How do I write extension modules for PyPy? ------------------------------------------ -See `Writing extension modules for PyPy`__. +See :doc:`extending`. -.. __: extending.html .. _how-fast-is-pypy: ------------------ How fast is PyPy? ----------------- This really depends on your code. For pure Python algorithmic code, it is very fast. For more typical Python programs we generally are 3 times the speed of Cpython 2.6 . -You might be interested in our `benchmarking site`_ and our -`jit documentation`_. +You might be interested in our `benchmarking site`_ and our +:doc:`jit documentation <rpython:jit/index>`. Note that the JIT has a very high warm-up cost, meaning that the programs are slow at the beginning. If you want to compare the timings @@ -116,11 +105,9 @@ with CPython, even relatively simple programs need to run *at least* one second, preferrably at least a few seconds. Large, complicated programs need even more time to warm-up the JIT. -.. _`benchmarking site`: http://speed.pypy.org +.. _benchmarking site: http://speed.pypy.org -.. _`jit documentation`: jit/index.html ---------------------------------------------------------------- Couldn't the JIT dump and reload already-compiled machine code? --------------------------------------------------------------- @@ -136,7 +123,7 @@ mapping addresses in the old (now-dead) process to addresses in the new process, including checking that all the previous assumptions about the (now-dead) object are still true about the new object. ------------------------------------------------------------ + How do I get into PyPy development? Can I come to sprints? ----------------------------------------------------------- @@ -147,15 +134,14 @@ developers. Newcomers should have some Python experience and read some of the PyPy documentation before coming to a sprint. Coming to a sprint is usually the best way to get into PyPy development. -If you get stuck or need advice, `contact us`_. IRC is +If you get stuck or need advice, :doc:`contact us <index>`. IRC is the most immediate way to get feedback (at least during some parts of the day; most PyPy developers are in Europe) and the `mailing list`_ is better for long discussions. -.. _`contact us`: index.html -.. _`mailing list`: http://python.org/mailman/listinfo/pypy-dev +.. _mailing list: http://python.org/mailman/listinfo/pypy-dev + -------------------------------------------------------------- OSError: ... cannot restore segment prot after reloc... Help? ------------------------------------------------------------- diff --git a/pypy/doc/getting-started-dev.rst b/pypy/doc/getting-started-dev.rst index 57e29c8369..6448e8a2d1 100644 --- a/pypy/doc/getting-started-dev.rst +++ b/pypy/doc/getting-started-dev.rst @@ -1,17 +1,16 @@ -=============================================== Getting Started with PyPy's Development Process =============================================== .. contents:: -.. _`start reading sources`: +.. _start-reading-sources: Where to start reading the sources ---------------------------------- PyPy is made from parts that are relatively independent of each other. You should start looking at the part that attracts you most (all paths are -relative to the PyPy top level directory). You may look at our `directory reference`_ +relative to the PyPy top level directory). You may look at our :doc:`directory reference <dir-reference>` or start off at one of the following points: * :source:`pypy/interpreter` contains the bytecode interpreter: bytecode dispatcher @@ -30,13 +29,11 @@ or start off at one of the following points: contains a modified version of the compiler package from CPython that fixes some bugs and is translatable. -* :source:`pypy/objspace/std` contains the `Standard object space`_. The main file +* :source:`pypy/objspace/std` contains the :ref:`Standard object space <standard-object-space>`. The main file is :source:`pypy/objspace/std/objspace.py`. For each type, the files ``xxxtype.py`` and ``xxxobject.py`` contain respectively the definition of the type and its (default) implementation. -.. _optionaltool: - Running PyPy's unit tests ------------------------- @@ -84,8 +81,9 @@ side, it's usually still faster than doing a full translation and running the regression test with the translated PyPy Python interpreter. -.. _`py.test testing tool`: http://pytest.org -.. _`py.test usage and invocations`: http://pytest.org/usage.html#usage +.. _py.test testing tool: http://pytest.org +.. _py.test usage and invocations: http://pytest.org/usage.html#usage + Special Introspection Features of the Untranslated Python Interpreter --------------------------------------------------------------------- @@ -94,8 +92,9 @@ If you are interested in the inner workings of the PyPy Python interpreter, there are some features of the untranslated Python interpreter that allow you to introspect its internals. + Interpreter-level console -+++++++++++++++++++++++++ +~~~~~~~~~~~~~~~~~~~~~~~~~ If you start an untranslated Python interpreter via:: @@ -104,7 +103,7 @@ If you start an untranslated Python interpreter via:: If you press <Ctrl-C> on the console you enter the interpreter-level console, a usual CPython console. You can then access internal objects of PyPy -(e.g. the `object space`_) and any variables you have created on the PyPy +(e.g. the :ref:`object space <objspace>`) and any variables you have created on the PyPy prompt with the prefix ``w_``:: >>>> a = 123 @@ -123,21 +122,18 @@ The mechanism works in both directions. If you define a variable with the ``w_`` >>>> l [1, 'abc'] -.. _`object space`: objspace.html - Note that the prompt of the interpreter-level console is only '>>>' since it runs on CPython level. If you want to return to PyPy, press <Ctrl-D> (under Linux) or <Ctrl-Z>, <Enter> (under Windows). You may be interested in reading more about the distinction between -`interpreter-level and app-level`_. +:ref:`interpreter-level and app-level <interpreter-level>`. -.. _`interpreter-level and app-level`: coding-guide.html#interpreter-level -.. _`trace example`: +.. _trace example: Tracing bytecode and operations on objects -++++++++++++++++++++++++++++++++++++++++++ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ You can use the trace object space to monitor the interpretation of bytecodes in connection with object space operations. To enable @@ -158,13 +154,15 @@ it, set ``__pytrace__=1`` on the interactive PyPy console:: |-13 RETURN_VALUE |- <<<< leave <inline>a = 1 + 2 @ 1 >>>> + Demos -------- +----- The `example-interpreter`_ repository contains an example interpreter written using the RPython translation toolchain. -.. _`example-interpreter`: https://bitbucket.org/pypy/example-interpreter +.. _example-interpreter: https://bitbucket.org/pypy/example-interpreter + Additional Tools for running (and hacking) PyPy ----------------------------------------------- @@ -173,8 +171,9 @@ We use some optional tools for developing PyPy. They are not required to run the basic tests or to get an interactive PyPy prompt but they help to understand and debug PyPy especially for the translation process. + graphviz & pygame for flow graph viewing (highly recommended) -+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ graphviz and pygame are both necessary if you want to look at generated flow graphs: @@ -183,8 +182,9 @@ want to look at generated flow graphs: pygame: http://www.pygame.org/download.shtml + py.test and the py lib -++++++++++++++++++++++ +~~~~~~~~~~~~~~~~~~~~~~ The `py.test testing tool`_ drives all our testing needs. @@ -194,26 +194,19 @@ writing, logging and some other support functionality. You don't necessarily need to install these two libraries because we also ship them inlined in the PyPy source tree. + Getting involved ---------------- PyPy employs an open development process. You are invited to join our -`pypy-dev mailing list`_ or look at the other `contact -possibilities`_. Usually we give out commit rights fairly liberally, so if you +`pypy-dev mailing list`_ or look at the other :ref:`contact +possibilities <contact>`. Usually we give out commit rights fairly liberally, so if you want to do something with PyPy, you can become a committer. We are also doing coding Sprints which are separately announced and often happen around Python conferences such as EuroPython or Pycon. Upcoming events are usually announced on `the blog`_. -.. _`the blog`: http://morepypy.blogspot.com -.. _`pypy-dev mailing list`: http://python.org/mailman/listinfo/pypy-dev -.. _`contact possibilities`: index.html - -.. _`py library`: http://pylib.org - -.. _Standard object space: objspace.html#the-standard-object-space -.. _mailing lists: index.html -.. _documentation: index.html#project-documentation -.. _unit tests: coding-guide.html#test-design +.. _the blog: http://morepypy.blogspot.com +.. _pypy-dev mailing list: http://python.org/mailman/listinfo/pypy-dev -.. _`directory reference`: index.html#pypy-directory-reference +.. _py library: http://pylib.org diff --git a/pypy/doc/getting-started-python.rst b/pypy/doc/getting-started-python.rst index 68a23bf035..d01016d2cf 100644 --- a/pypy/doc/getting-started-python.rst +++ b/pypy/doc/getting-started-python.rst @@ -1,4 +1,3 @@ -============================================== Getting Started with PyPy's Python Interpreter ============================================== @@ -7,35 +6,33 @@ Getting Started with PyPy's Python Interpreter PyPy's Python interpreter is a very compliant Python interpreter implemented in RPython. When compiled, it passes most of -`CPythons core language regression tests`_ and comes with many of the extension +`CPython's core language regression tests`_ and comes with many of the extension modules included in the standard library including ``ctypes``. It can run large libraries such as Django_ and Twisted_. There are some small behavioral -differences with CPython and some missing extensions, for details see `CPython -differences`_. +differences with CPython and some missing extensions, for details see :doc:`cpython_differences` + +To actually use PyPy's Python interpreter, the first thing to do is to +`download a pre-built PyPy`_ for your architecture. +.. _CPython's core language regression tests: http://buildbot.pypy.org/summary?category=applevel&branch=%3Ctrunk%3E .. _Django: http://djangoproject.com .. _Twisted: http://twistedmatrix.com +.. _download a pre-built PyPy: http://pypy.org/download.html -.. _`CPython differences`: cpython_differences.html -To actually use PyPy's Python interpreter, the first thing to do is to -`download a pre-built PyPy`_ for your architecture. - -.. _`download a pre-built PyPy`: http://pypy.org/download.html +.. _translate-pypy: Translating the PyPy Python interpreter --------------------------------------- -(**Note**: for some hints on how to translate the Python interpreter under -Windows, see the `windows document`_) - -.. _`windows document`: windows.html +.. note:: For some hints on how to translate the Python interpreter under + Windows, see the :doc:`windows document <rpython:windows>`. You can translate the whole of PyPy's Python interpreter to low level C code, or `CLI code`_. If you intend to build using gcc, check to make sure that the version you have is not 4.2 or you will run into `this bug`_. -.. _`this bug`: https://bugs.launchpad.net/ubuntu/+source/gcc-4.2/+bug/187391 +.. _this bug: https://bugs.launchpad.net/ubuntu/+source/gcc-4.2/+bug/187391 1. First `download a pre-built PyPy`_ for your architecture which you will use to translate your Python interpreter. It is, of course, possible to @@ -75,7 +72,7 @@ the version you have is not 4.2 or you will run into `this bug`_. memory on a 32-bit machine and 4GB on a 64-bit machine. If your memory resources are constrained, or your machine is slow you might want to pick the - `optimization level`_ `1` in the next step. A level of + :doc:`optimization level <config/opt>` `1` in the next step. A level of `2` or `3` or `jit` gives much better results, though. But if all you want to do is to test that some new feature that you just wrote translates, level 1 is enough. @@ -90,12 +87,10 @@ the version you have is not 4.2 or you will run into `this bug`_. cd pypy/goal python ../../rpython/bin/rpython --opt=jit targetpypystandalone.py - possibly replacing ``--opt=jit`` with another `optimization level`_ + possibly replacing ``--opt=jit`` with another :doc:`optimization level <config/opt>` of your choice like ``--opt=2`` if you do not want to include the JIT compiler, which makes the Python interpreter much slower. -.. _`optimization level`: config/opt.html - If everything works correctly this will create an executable ``pypy-c`` in the current directory. Type ``pypy-c --help`` to see the options it supports - mainly the same basic @@ -128,28 +123,26 @@ Installation_ below. The ``translate.py`` script takes a very large number of options controlling what to translate and how. See ``translate.py -h``. The default options should be suitable for mostly everybody by now. -Find a more detailed description of the various options in our `configuration -sections`_. +Find a more detailed description of the various options in our :doc:`configuration +sections <config/index>`. -.. _`configuration sections`: config/index.html Translating with non-standard options -+++++++++++++++++++++++++++++++++++++ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ It is possible to have non-standard features enabled for translation, but they are not really tested any more. Look, for example, at the -`objspace proxies`_ document. +:doc:`objspace proxies <objspace-proxies>` document. -.. _`objspace proxies`: objspace-proxies.html -.. _`CLI code`: +.. _CLI code: Translating using the CLI backend -+++++++++++++++++++++++++++++++++ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -**Note: the CLI backend is no longer maintained** +.. note:: The CLI backend is no longer maintained. -To create a standalone .NET executable using the `CLI backend`_:: +To create a standalone .NET executable using the :doc:`CLI backend <rpython:cli-backend>`:: ./translate.py --backend=cli targetpypystandalone.py @@ -178,36 +171,40 @@ will automatically use its ``ilasm2`` tool to assemble the executables. To try out the experimental .NET integration, check the documentation of the -clr_ module. +:doc:`clr <clr-module>` module. + +.. _Mono: http://www.mono-project.com/Main_Page -.. not working now: - .. _`JVM code`: +.. _JVM code: - Translating using the JVM backend - +++++++++++++++++++++++++++++++++ +Translating using the JVM backend +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - To create a standalone JVM executable:: +.. note:: The JVM backend is no longer maintained. - ./translate.py --backend=jvm targetpypystandalone.py +To create a standalone JVM executable:: - This will create a jar file ``pypy-jvm.jar`` as well as a convenience - script ``pypy-jvm`` for executing it. To try it out, simply run - ``./pypy-jvm``:: + ./translate.py --backend=jvm targetpypystandalone.py - $ ./pypy-jvm - Python 2.7.0 (61ef2a11b56a, Mar 02 2011, 03:00:11) - [PyPy 1.6.0] on linux2 - Type "help", "copyright", "credits" or "license" for more information. - And now for something completely different: ``# assert did not crash'' - >>>> +This will create a jar file ``pypy-jvm.jar`` as well as a convenience +script ``pypy-jvm`` for executing it. To try it out, simply run +``./pypy-jvm``:: + + $ ./pypy-jvm + Python 2.7.0 (61ef2a11b56a, Mar 02 2011, 03:00:11) + [PyPy 1.6.0] on linux2 + Type "help", "copyright", "credits" or "license" for more information. + And now for something completely different: ``# assert did not crash'' + >>>> + +Alternatively, you can run it using ``java -jar pypy-jvm.jar``. At the moment +the executable does not provide any interesting features, like integration with +Java. - Alternatively, you can run it using ``java -jar pypy-jvm.jar``. At the moment - the executable does not provide any interesting features, like integration with - Java. Installation -++++++++++++ +~~~~~~~~~~~~ A prebuilt ``pypy-c`` can be installed in a standard location like ``/usr/local/bin``, although some details of this process are still in @@ -244,13 +241,14 @@ and then attempt to continue normally. If the default path is usable, most code will be fine. However, the ``sys.prefix`` will be unset and some existing libraries assume that this is never the case. -.. _`pyinteractive.py interpreter`: + +.. _pyinteractive.py interpreter: Running the Python Interpreter Without Translation -------------------------------------------------- The pyinteractive.py interpreter -++++++++++++++++++++++++++++++++ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To start interpreting Python with PyPy, install a C compiler that is supported by distutils and use Python 2.5 or greater to run PyPy:: @@ -276,8 +274,9 @@ default is 50000, which is far too many to run in a non-translated PyPy version (i.e. when PyPy's interpreter itself is being interpreted by CPython). + pyinteractive.py options -++++++++++++++++++++++++ +~~~~~~~~~~~~~~~~~~~~~~~~ To list the PyPy interpreter command line options, type:: @@ -295,12 +294,5 @@ script name on the command line:: python pyinteractive.py ../../lib-python/2.7/test/pystone.py 10 -See our `configuration sections`_ for details about what all the commandline +See our :doc:`configuration sections <config/index>` for details about what all the commandline options do. - - -.. _Mono: http://www.mono-project.com/Main_Page -.. _`CLI backend`: cli-backend.html -.. _`Boehm-Demers-Weiser garbage collector`: http://www.hpl.hp.com/personal/Hans_Boehm/gc/ -.. _clr: clr-module.html -.. _`CPythons core language regression tests`: http://buildbot.pypy.org/summary?category=applevel&branch=%3Ctrunk%3E diff --git a/pypy/doc/getting-started.rst b/pypy/doc/getting-started.rst index 78e374d6ad..35373a7fcd 100644 --- a/pypy/doc/getting-started.rst +++ b/pypy/doc/getting-started.rst @@ -1,13 +1,11 @@ -================================== Getting Started ================================== .. contents:: -.. _howtopypy: What is PyPy ? -============== +-------------- In common parlance, PyPy has been used to mean two things. The first is the `RPython translation toolchain`_, which is a framework for generating @@ -27,28 +25,32 @@ We target a large variety of platforms, small and large, by providing a compiler toolsuite that can produce custom Python versions. Platform, memory and threading models, as well as the JIT compiler itself, are aspects of the translation process - as opposed to encoding low level details into the -language implementation itself. `more...`_ +language implementation itself. :doc:`more... <architecture>` .. _Python: http://docs.python.org/reference/ -.. _`RPython translation toolchain`: translation.html -.. _`more...`: architecture.html +.. _RPython translation toolchain: translation.html +.. _more...: architecture.html + Just the facts -============== +-------------- + +.. _prebuilt-pypy: Download a pre-built PyPy -------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~ The quickest way to start using PyPy is to download a prebuilt binary for your OS and architecture. You can either use the `most recent release`_ or one of our `development nightly build`_. Please note that the nightly builds are not guaranteed to be as stable as official releases, use them at your own risk. -.. _`most recent release`: http://pypy.org/download.html -.. _`development nightly build`: http://buildbot.pypy.org/nightly/trunk/ +.. _most recent release: http://pypy.org/download.html +.. _development nightly build: http://buildbot.pypy.org/nightly/trunk/ + Installing PyPy ---------------- +~~~~~~~~~~~~~~~ PyPy is ready to be executed as soon as you unpack the tarball or the zip file, with no need to install it in any specific location:: @@ -84,8 +86,9 @@ install distribute_ and pip_: 3rd party libraries will be installed in ``pypy-2.0-beta1/site-packages``, and the scripts in ``pypy-2.0-beta1/bin``. + Installing using virtualenv ---------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~~ It is often convenient to run pypy inside a virtualenv. To do this you need a recent version of virtualenv -- 1.6.1 or greater. You can @@ -100,12 +103,12 @@ checkout:: Note that bin/python is now a symlink to bin/pypy. -.. _`distribute`: http://www.python-distribute.org/ -.. _`pip`: http://pypi.python.org/pypi/pip +.. _distribute: http://www.python-distribute.org/ +.. _pip: http://pypi.python.org/pypi/pip Clone the repository --------------------- +~~~~~~~~~~~~~~~~~~~~ If you prefer to `compile PyPy by yourself`_, or if you want to modify it, you will need to obtain a copy of the sources. This can be done either by @@ -113,7 +116,7 @@ will need to obtain a copy of the sources. This can be done either by repository using mercurial. We suggest using mercurial if one wants to access the current development. -.. _`downloading them from the download page`: http://pypy.org/download.html +.. _downloading them from the download page: http://pypy.org/download.html You must issue the following command on your command line, DOS box, or terminal:: @@ -124,7 +127,7 @@ This will clone the repository and place it into a directory named ``pypy``, and will get you the PyPy source in ``pypy/pypy`` and documentation files in ``pypy/pypy/doc``. We try to ensure that the tip is always stable, but it might -occasionally be broken. You may want to check out `our nightly tests:`_ +occasionally be broken. You may want to check out `our nightly tests`_: find a revision (12-chars alphanumeric string, e.g. "963e808156b3") that passed at least the ``{linux32}`` tests (corresponding to a ``+`` sign on the @@ -135,45 +138,40 @@ using:: where XXXXX is the revision id. +.. _compile PyPy by yourself: getting-started-python.html +.. _our nightly tests: http://buildbot.pypy.org/summary?branch=<trunk> -.. _`compile PyPy by yourself`: getting-started-python.html -.. _`our nightly tests:`: http://buildbot.pypy.org/summary?branch=<trunk> Where to go from here -====================== +---------------------- After you successfully manage to get PyPy's source you can read more about: - - `Building and using PyPy's Python interpreter`_ - - `Learning more about the RPython toolchain and how to develop (with) PyPy`_ + - :doc:`Building and using PyPy's Python interpreter <getting-started-python>` + - :doc:`Learning more about the RPython toolchain and how to develop (with) PyPy <getting-started-dev>` - `Tutorial for how to write an interpreter with the RPython toolchain and make it fast`_ - `Look at our benchmark results`_ -.. _`Building and using PyPy's Python interpreter`: getting-started-python.html -.. _`Learning more about the RPython toolchain and how to develop (with) PyPy`: getting-started-dev.html -.. _`Tutorial for how to write an interpreter with the RPython toolchain and make it fast`: http://morepypy.blogspot.com/2011/04/tutorial-writing-interpreter-with-pypy.html -.. _`Look at our benchmark results`: http://speed.pypy.org +.. _Tutorial for how to write an interpreter with the RPython toolchain and make it fast: http://morepypy.blogspot.com/2011/04/tutorial-writing-interpreter-with-pypy.html +.. _Look at our benchmark results: http://speed.pypy.org -.. _setuptools: http://pypi.python.org/pypi/setuptools Understanding PyPy's architecture ---------------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ For in-depth information about architecture and coding documentation -head over to the `documentation section`_ where you'll find lots of +head over to the :doc:`documentation section <project-documentation>` where you'll find lots of interesting information. Additionally, in true hacker spirit, you -may just `start reading sources`_ . +may just :ref:`start reading sources <start-reading-sources>`. -.. _`documentation section`: index.html#project-documentation -.. _`start reading sources`: getting-started-dev.html#start-reading-sources Filing bugs or feature requests -------------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ You may file `bug reports`_ on our issue tracker which is also accessible through the 'issues' top menu of the PyPy website. `Using the development tracker`_ has more detailed information on specific features of the tracker. -.. _`Using the development tracker`: coding-guide.html#using-development-tracker +.. _Using the development tracker: coding-guide.html#using-development-tracker .. _bug reports: https://bugs.pypy.org/ diff --git a/pypy/doc/how-to-contribute.rst b/pypy/doc/how-to-contribute.rst index 02f18e4f81..be8426de36 100644 --- a/pypy/doc/how-to-contribute.rst +++ b/pypy/doc/how-to-contribute.rst @@ -1,11 +1,12 @@ How to contribute to PyPy -------------------------- +========================= This page describes how to contribute to the PyPy project. The first thing to remember is that PyPy project is very different than most projects out there. It's also different from a classic compiler project, so academic courses about compilers often don't apply or lead in the wrong direction. + Don't just hack --------------- @@ -14,6 +15,7 @@ The first and most important rule how not to contribute to PyPy is -- build times are large and PyPy has very thick layer separation which make it harder to "just hack a feature". + Test driven development ----------------------- @@ -25,7 +27,8 @@ not one of those. You may consider familiarizing yourself with `pytest`_, since this is a tool we use for tests. This leads to the next issue: -.. _`pytest`: http://pytest.org/ +.. _pytest: http://pytest.org/ + Layers ------ @@ -63,7 +66,7 @@ with a JIT compiler, use ``--viewloops`` option. **module** directory contains extension modules written in RPython * **rpython compiler** that resides in ``rpython/annotator`` and - ``rpython/rtyper`` directories. Consult `introduction to RPython`_ for + ``rpython/rtyper`` directories. Consult :doc:`introduction to RPython <getting-started-dev>` for further reading * **JIT generator** lives in ``rpython/jit`` directory. optimizations live @@ -76,5 +79,3 @@ with a JIT compiler, use ``--viewloops`` option. The rest of directories serve specific niche goal and are unlikely a good entry point. - -.. _`introduction to RPython`: getting-started-dev.rst diff --git a/pypy/doc/how-to-release.rst b/pypy/doc/how-to-release.rst index 59f29473eb..f9db0a3616 100644 --- a/pypy/doc/how-to-release.rst +++ b/pypy/doc/how-to-release.rst @@ -1,9 +1,8 @@ - Making a PyPy Release -======================= +===================== Overview ---------- +-------- As a meta rule setting up issues in the tracker for items here may help not forgetting things. A set of todo files may also work. @@ -12,8 +11,9 @@ Check and prioritize all issues for the release, postpone some if necessary, create new issues also as necessary. An important thing is to get the documentation into an up-to-date state! + Release Steps ----------------- +------------- * at code freeze make a release branch using release-x.x.x in mercurial IMPORTANT: bump the diff --git a/pypy/doc/index-report.rst b/pypy/doc/index-report.rst index 30e47908aa..1f87a0dbbd 100644 --- a/pypy/doc/index-report.rst +++ b/pypy/doc/index-report.rst @@ -2,27 +2,26 @@ Some of these reports are interesting for historical reasons only. - -============================================ PyPy - Overview over the EU-reports -============================================ +=================================== -Below reports summarize and discuss research and development results -of the PyPy project during the EU funding period (Dez 2004 - March 2007). +Below reports summarize and discuss research and development results +of the PyPy project during the EU funding period (Dez 2004 - March 2007). They also are very good documentation if you'd like to know in more -detail about motivation and implementation of the various parts +detail about motivation and implementation of the various parts and aspects of PyPy. Feel free to send questions or comments to `pypy-dev`_, the development list. + Reports of 2007 -=============== +--------------- The `PyPy EU Final Activity Report`_ summarizes the 28 month EU project -period (Dec 2004-March 2007) on technical, scientific and community levels. -You do not need prior knowledge about PyPy but some technical knowledge about -computer language implementations is helpful. The report contains reflections -and recommendations which might be interesting for other project aiming -at funded Open Source research. *(2007-05-11)* +period (Dec 2004-March 2007) on technical, scientific and community levels. +You do not need prior knowledge about PyPy but some technical knowledge about +computer language implementations is helpful. The report contains reflections +and recommendations which might be interesting for other project aiming +at funded Open Source research. *(2007-05-11)* `D09.1 Constraint Solving and Semantic Web`_ is a report about PyPy's logic programming and constraint solving features, as well as the work going on to @@ -75,7 +74,7 @@ a report about our high-level backends and our several validation prototypes: an information flow security prototype, a distribution prototype and a persistence proof-of-concept. *(2007-03-22)* -`D14.2 Tutorials and Guide Through the PyPy Source Code`_ is +`D14.2 Tutorials and Guide Through the PyPy Source Code`_ is a report about the steps we have taken to make the project approachable for newcomers. *(2007-03-22)* @@ -94,24 +93,22 @@ PyPy's optimization efforts, garbage collectors and massive parallelism (stackless) features. This report refers to the paper `PyPy's approach to virtual machine construction`_. *(2007-02-28)* - - -.. _`py-lib`: http://pylib.org/ -.. _`py.test`: http://pytest.org/ +.. _py-lib: http://pylib.org/ +.. _py.test: http://pytest.org/ .. _codespeak: http://codespeak.net/ -.. _`pypy-dev`: http://python.org/mailman/listinfo/pypy-dev +.. _pypy-dev: http://python.org/mailman/listinfo/pypy-dev Reports of 2006 -=============== +--------------- `D14.3 Report about Milestone/Phase 2`_ is the final report about -the second phase of the EU project, summarizing and detailing technical, -research, dissemination and community aspects. Feedback is very welcome! +the second phase of the EU project, summarizing and detailing technical, +research, dissemination and community aspects. Feedback is very welcome! Reports of 2005 -=============== +--------------- `D04.1 Partial Python Implementation`_ contains details about the 0.6 release. All the content can be found in the regular documentation section. @@ -140,35 +137,33 @@ properties into our interpreter during the translation process. `D14.1 Report about Milestone/Phase 1`_ describes what happened in the PyPy project during the first year of EU funding (December 2004 - December 2005) -.. _`PyPy EU Final Activity Report`: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/PYPY-EU-Final-Activity-Report.pdf -.. _`D01.2-4 Project Organization`: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D01.2-4_Project_Organization-2007-03-28.pdf -.. _`D02.1 Development Tools and Website`: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D02.1_Development_Tools_and_Website-2007-03-21.pdf -.. _`D02.2 Release Scheme`: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D02.2_Release_Scheme-2007-03-30.pdf -.. _`D02.3 Testing Tool`: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D02.3_Testing_Framework-2007-03-23.pdf -.. _`D03.1 Extension Compiler`: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D03.1_Extension_Compiler-2007-03-21.pdf -.. _`D04.1 Partial Python Implementation`: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D04.1_Partial_Python_Implementation_on_top_of_CPython.pdf -.. _`D04.2 Complete Python Implementation`: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D04.2_Complete_Python_Implementation_on_top_of_CPython.pdf -.. _`D04.3 Parser and Bytecode Compiler`: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D04.3_Report_about_the_parser_and_bytecode_compiler.pdf -.. _`D04.4 PyPy as a Research Tool`: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D04.4_Release_PyPy_as_a_research_tool.pdf -.. _`D05.1 Compiling Dynamic Language Implementations`: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D05.1_Publish_on_translating_a_very-high-level_description.pdf -.. _`D05.2 A Compiled Version of PyPy`: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D05.2_A_compiled,_self-contained_version_of_PyPy.pdf -.. _`D05.3 Implementation with Translation Aspects`: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D05.3_Publish_on_implementation_with_translation_aspects.pdf -.. _`D05.4 Encapsulating Low Level Aspects`: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D05.4_Publish_on_encapsulating_low_level_language_aspects.pdf -.. _`D06.1 Core Object Optimization Results`: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D06.1_Core_Optimizations-2007-04-30.pdf -.. _`D07.1 Massive Parallelism and Translation Aspects`: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D07.1_Massive_Parallelism_and_Translation_Aspects-2007-02-28.pdf -.. _`D08.2 JIT Compiler Architecture`: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D08.2_JIT_Compiler_Architecture-2007-05-01.pdf -.. _`D08.1 JIT Compiler Release`: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D08.1_JIT_Compiler_Release-2007-04-30.pdf -.. _`D09.1 Constraint Solving and Semantic Web`: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D09.1_Constraint_Solving_and_Semantic_Web-2007-05-11.pdf -.. _`D10.1 Aspect-Oriented, Design-by-Contract Programming and RPython static checking`: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D10.1_Aspect_Oriented_Programming_in_PyPy-2007-03-22.pdf -.. _`D11.1 PyPy for Embedded Devices`: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D11.1_PyPy_for_Embedded_Devices-2007-03-26.pdf -.. _`D12.1 High-Level-Backends and Feature Prototypes`: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D12.1_H-L-Backends_and_Feature_Prototypes-2007-03-22.pdf -.. _`D13.1 Integration and Configuration`: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D13.1_Integration_and_Configuration-2007-03-30.pdf -.. _`D14.1 Report about Milestone/Phase 1`: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D14.1_Report_about_Milestone_Phase_1.pdf -.. _`D14.2 Tutorials and Guide Through the PyPy Source Code`: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D14.2_Tutorials_and_Guide_Through_the_PyPy_Source_Code-2007-03-22.pdf -.. _`D14.3 Report about Milestone/Phase 2`: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D14.3_Report_about_Milestone_Phase_2-final-2006-08-03.pdf -.. _`D14.4 PyPy-1.0 Milestone report`: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D14.4_Report_About_Milestone_Phase_3-2007-05-01.pdf -.. _`D14.5 Documentation of the development process`: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D14.5_Documentation_of_the_development_process-2007-03-30.pdf - - - -.. _`PyPy's approach to virtual machine construction`: https://bitbucket.org/pypy/extradoc/raw/tip/talk/dls2006/pypy-vm-construction.pdf +.. _PyPy EU Final Activity Report: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/PYPY-EU-Final-Activity-Report.pdf +.. _D01.2-4 Project Organization: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D01.2-4_Project_Organization-2007-03-28.pdf +.. _D02.1 Development Tools and Website: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D02.1_Development_Tools_and_Website-2007-03-21.pdf +.. _D02.2 Release Scheme: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D02.2_Release_Scheme-2007-03-30.pdf +.. _D02.3 Testing Tool: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D02.3_Testing_Framework-2007-03-23.pdf +.. _D03.1 Extension Compiler: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D03.1_Extension_Compiler-2007-03-21.pdf +.. _D04.1 Partial Python Implementation: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D04.1_Partial_Python_Implementation_on_top_of_CPython.pdf +.. _D04.2 Complete Python Implementation: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D04.2_Complete_Python_Implementation_on_top_of_CPython.pdf +.. _D04.3 Parser and Bytecode Compiler: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D04.3_Report_about_the_parser_and_bytecode_compiler.pdf +.. _D04.4 PyPy as a Research Tool: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D04.4_Release_PyPy_as_a_research_tool.pdf +.. _D05.1 Compiling Dynamic Language Implementations: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D05.1_Publish_on_translating_a_very-high-level_description.pdf +.. _D05.2 A Compiled Version of PyPy: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D05.2_A_compiled,_self-contained_version_of_PyPy.pdf +.. _D05.3 Implementation with Translation Aspects: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D05.3_Publish_on_implementation_with_translation_aspects.pdf +.. _D05.4 Encapsulating Low Level Aspects: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D05.4_Publish_on_encapsulating_low_level_language_aspects.pdf +.. _D06.1 Core Object Optimization Results: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D06.1_Core_Optimizations-2007-04-30.pdf +.. _D07.1 Massive Parallelism and Translation Aspects: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D07.1_Massive_Parallelism_and_Translation_Aspects-2007-02-28.pdf +.. _D08.2 JIT Compiler Architecture: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D08.2_JIT_Compiler_Architecture-2007-05-01.pdf +.. _D08.1 JIT Compiler Release: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D08.1_JIT_Compiler_Release-2007-04-30.pdf +.. _D09.1 Constraint Solving and Semantic Web: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D09.1_Constraint_Solving_and_Semantic_Web-2007-05-11.pdf +.. _D10.1 Aspect-Oriented, Design-by-Contract Programming and RPython static checking: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D10.1_Aspect_Oriented_Programming_in_PyPy-2007-03-22.pdf +.. _D11.1 PyPy for Embedded Devices: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D11.1_PyPy_for_Embedded_Devices-2007-03-26.pdf +.. _D12.1 High-Level-Backends and Feature Prototypes: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D12.1_H-L-Backends_and_Feature_Prototypes-2007-03-22.pdf +.. _D13.1 Integration and Configuration: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D13.1_Integration_and_Configuration-2007-03-30.pdf +.. _D14.1 Report about Milestone/Phase 1: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D14.1_Report_about_Milestone_Phase_1.pdf +.. _D14.2 Tutorials and Guide Through the PyPy Source Code: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D14.2_Tutorials_and_Guide_Through_the_PyPy_Source_Code-2007-03-22.pdf +.. _D14.3 Report about Milestone/Phase 2: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D14.3_Report_about_Milestone_Phase_2-final-2006-08-03.pdf +.. _D14.4 PyPy-1.0 Milestone report: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D14.4_Report_About_Milestone_Phase_3-2007-05-01.pdf +.. _D14.5 Documentation of the development process: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D14.5_Documentation_of_the_development_process-2007-03-30.pdf + +.. _PyPy's approach to virtual machine construction: https://bitbucket.org/pypy/extradoc/raw/tip/talk/dls2006/pypy-vm-construction.pdf diff --git a/pypy/doc/index.rst b/pypy/doc/index.rst index 679ae75b4d..2abd606543 100644 --- a/pypy/doc/index.rst +++ b/pypy/doc/index.rst @@ -1,4 +1,3 @@ - Welcome to PyPy =============== @@ -6,75 +5,99 @@ The PyPy project aims to produce a flexible and fast Python_ implementation. This page documents the development of the PyPy project itself. If you don't know what PyPy is, consult the `PyPy website`_. If you just want to use PyPy, consult the `download`_ page -and the `getting started with pypy`_ documents. If you want to help +and the :doc:`getting-started-python` documents. If you want to help develop PyPy -- keep reading! PyPy is written in a language called `RPython`_, which is suitable for writing dynamic language interpreters (and not much else). RPython is a subset of Python and is itself written in Python. If you'd like to -learn more about RPython, `Starting with RPython`_ should provide a +learn more about RPython, `XXXX` should provide a reasonable overview. -**If you would like to contribute to PyPy**, please read `how to -contribute`_ first. PyPy's development style is somewhat different to +**If you would like to contribute to PyPy**, please read :doc:`how to +contribute <how-to-contribute>` first. PyPy's development style is somewhat different to that of many other software projects and it often surprises newcomers. What is **not** necessary is an academic background from university in writing compilers -- much of it does not apply to PyPy any way. All of the documentation and source code is available under the MIT license, -unless otherwise specified. Consult `LICENSE`_ +unless otherwise specified. Consult :source:`LICENSE`. + +.. _Python: http://python.org/ +.. _download: http://pypy.org/download.html +.. _PyPy website: http://pypy.org/ +.. _RPython: http://rpython.readthedocs.org/ + +.. toctree:: + :hidden: + + getting-started-python + how-to-contribute -.. _`download`: http://pypy.org/download.html -.. _`getting started with pypy`: getting-started-python.html -.. _`RPython`: coding-guide.html#RPython -.. _`Starting with RPython`: getting-started-dev.html -.. _`how to contribute`: how-to-contribute.html -.. _`PyPy website`: http://pypy.org -.. _`LICENSE`: https://bitbucket.org/pypy/pypy/src/default/LICENSE Index of various topics: -======================== +------------------------ -* `Getting started`_: how to install and run the PyPy Python interpreter +* :doc:`getting-started`: how to install and run the PyPy Python interpreter -* `FAQ`_: some frequently asked questions. +* :doc:`FAQ <faq>`: some frequently asked questions. * `Release 2.0 beta 2`_: the latest official release * `PyPy Blog`_: news and status info about PyPy -* `Papers`_: Academic papers, talks, and related projects +* :doc:`Papers <extradoc>`: Academic papers, talks, and related projects * `speed.pypy.org`_: Daily benchmarks of how fast PyPy is -* `potential project ideas`_: In case you want to get your feet wet... +* :doc:`project-ideas`: In case you want to get your feet wet... -* `more stuff`_: this is a collection of documentation that's there, but not +* :doc:`More stuff <project-documentation>`: this is a collection of documentation that's there, but not particularly organized +.. _PyPy blog: http://morepypy.blogspot.com/ +.. _Release 2.0 beta 2: http://pypy.org/download.html +.. _speed.pypy.org: http://speed.pypy.org + +.. toctree:: + :hidden: + + getting-started + faq + extradoc + project-ideas + project-documentation + + Documentation for the PyPy Python Interpreter -============================================= +--------------------------------------------- New features of PyPy's Python Interpreter and Translation Framework: - * `Differences between PyPy and CPython`_ - * `What PyPy can do for your objects`_ - transparent proxy documentation - * `Continulets and greenlets`_ - documentation about stackless features - * `JIT hooks`_ - * `Sandboxing Python code`_ - * `Garbage collection environment variables`_ + * :doc:`cpython_differences` + * :doc:`objspace-proxies` - transparent proxy documentation + * :doc:`Continulets and greenlets <stackless>` - documentation about stackless features + * :doc:`jit-hooks` + * :doc:`sandbox` + * :doc:`Garbage collection environment variables <gc_info>` -.. _`Differences between PyPy and CPython`: cpython_differences.html -.. _`What PyPy can do for your objects`: objspace-proxies.html -.. _`Continulets and greenlets`: stackless.html -.. _`JIT hooks`: jit-hooks.html -.. _`Sandboxing Python code`: sandbox.html -.. _`Garbage collection environment variables`: gc_info.html +.. toctree:: + :hidden: + + cpython_differences + objspace-proxies + stackless + jit-hooks + sandbox + gc_info + + +.. _contact: Mailing lists, bug tracker, IRC channel -============================================= +--------------------------------------------- * `Development mailing list`_: development and conceptual discussions. @@ -86,30 +109,19 @@ Mailing lists, bug tracker, IRC channel * **IRC channel #pypy on freenode**: Many of the core developers are hanging out at #pypy on irc.freenode.net. You are welcome to join and ask questions - (if they are not already developed in the FAQ_). + (if they are not already developed in the :doc:`FAQ <faq>`). You can find logs of the channel here_. +.. _development mailing list: http://python.org/mailman/listinfo/pypy-dev +.. _Mercurial commit mailing list: http://python.org/mailman/listinfo/pypy-commit +.. _development bug/feature tracker: https://bugs.pypy.org +.. _here: http://tismerysoft.de/pypy/irc-logs/pypy + + Meeting PyPy developers -======================= +----------------------- The PyPy developers are organizing sprints and presenting results at conferences all year round. They will be happy to meet in person with anyone interested in the project. Watch out for sprint announcements on the `development mailing list`_. - -.. _Python: http://docs.python.org/index.html -.. _`more...`: architecture.html#mission-statement -.. _`PyPy blog`: http://morepypy.blogspot.com/ -.. _`development bug/feature tracker`: https://bugs.pypy.org -.. _here: http://tismerysoft.de/pypy/irc-logs/pypy -.. _`Mercurial commit mailing list`: http://python.org/mailman/listinfo/pypy-commit -.. _`development mailing list`: http://python.org/mailman/listinfo/pypy-dev -.. _`FAQ`: faq.html -.. _`Getting Started`: getting-started.html -.. _`Papers`: extradoc.html -.. _`Videos`: video-index.html -.. _`Release 2.0 beta 2`: http://pypy.org/download.html -.. _`speed.pypy.org`: http://speed.pypy.org -.. _`RPython toolchain`: translation.html -.. _`potential project ideas`: project-ideas.html -.. _`more stuff`: project-documentation.html diff --git a/pypy/doc/interpreter-optimizations.rst b/pypy/doc/interpreter-optimizations.rst index 1f70fdbd01..6dbc8f89bc 100644 --- a/pypy/doc/interpreter-optimizations.rst +++ b/pypy/doc/interpreter-optimizations.rst @@ -1,11 +1,10 @@ -================================== Standard Interpreter Optimizations ================================== -.. contents:: Contents +.. contents:: Introduction -============ +------------ One of the advantages -- indeed, one of the motivating goals -- of the PyPy standard interpreter (compared to CPython) is that of increased flexibility and @@ -28,11 +27,12 @@ them. And they are fun too! .. describe other optimizations! + Object Optimizations -==================== +-------------------- Integer Optimizations ---------------------- +~~~~~~~~~~~~~~~~~~~~~ Caching Small Integers ++++++++++++++++++++++ @@ -45,6 +45,7 @@ be retrieved from the cache. This option is disabled by default, you can enable this feature with the :config:`objspace.std.withprebuiltint` option. + Integers as Tagged Pointers +++++++++++++++++++++++++++ @@ -57,8 +58,9 @@ time and memory. You can enable this feature with the :config:`objspace.std.withsmalllong` option. + Dictionary Optimizations ------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~ Multi-Dicts +++++++++++ @@ -75,6 +77,7 @@ implementations for various purposes (see below). This is now the default implementation of dictionaries in the Python interpreter. + Sharing Dicts +++++++++++++ @@ -97,7 +100,7 @@ with the :config:`objspace.std.withmapdict` option. List Optimizations ------------------- +~~~~~~~~~~~~~~~~~~ Range-Lists +++++++++++ @@ -116,8 +119,7 @@ option. User Class Optimizations ------------------------- - +~~~~~~~~~~~~~~~~~~~~~~~~ Method Caching ++++++++++++++ @@ -134,13 +136,14 @@ as long as the instance did not shadow any of its classes attributes. You can enable this feature with the :config:`objspace.std.withmethodcache` option. + Interpreter Optimizations -========================= +------------------------- Special Bytecodes ------------------ +~~~~~~~~~~~~~~~~~ -.. _`lookup method call method`: +.. _lookup method call method: LOOKUP_METHOD & CALL_METHOD +++++++++++++++++++++++++++ @@ -203,8 +206,9 @@ option. .. more here? + Overall Effects -=============== +--------------- The impact these various optimizations have on performance unsurprisingly depends on the program being run. Using the default multi-dict implementation that diff --git a/pypy/doc/interpreter.rst b/pypy/doc/interpreter.rst index bf3c09b41c..ec86b5c41d 100644 --- a/pypy/doc/interpreter.rst +++ b/pypy/doc/interpreter.rst @@ -1,13 +1,11 @@ -=================================== Bytecode Interpreter -=================================== +==================== .. contents:: - Introduction and Overview -=============================== +------------------------- This document describes the implementation of PyPy's Bytecode Interpreter and related Virtual Machine functionalities. @@ -17,8 +15,8 @@ Virtual Machine: It processes code objects parsed and compiled from Python source code. It is implemented in the :source:`pypy/interpreter/` directory. People familiar with the CPython implementation will easily recognize similar concepts there. The major differences are the overall usage of -the `object space`_ indirection to perform operations on objects, and -the organization of the built-in modules (described `here`_). +the :doc:`object space <objspace>` indirection to perform operations on objects, and +the organization of the built-in modules (described :ref:`here <modules>`). Code objects are a nicely preprocessed, structured representation of source code, and their main content is *bytecode*. We use the same @@ -56,22 +54,22 @@ from a stack. The bytecode interpreter is only responsible for implementing control flow and pushing and pulling black box objects to and from this value stack. The bytecode interpreter does not know how to perform operations on those black box -(`wrapped`_) objects for which it delegates to the `object -space`_. In order to implement a conditional branch in a program's +(:ref:`wrapped <wrapped>`) objects for which it delegates to the :doc:`object +space <objspace>`. In order to implement a conditional branch in a program's execution, however, it needs to gain minimal knowledge about a wrapped object. Thus, each object space has to offer a ``is_true(w_obj)`` operation which returns an interpreter-level boolean value. For the understanding of the interpreter's inner workings it -is crucial to recognize the concepts of `interpreter-level and -application-level`_ code. In short, interpreter-level is executed +is crucial to recognize the concepts of :ref:`interpreter-level and +application-level <interpreter-level>` code. In short, interpreter-level is executed directly on the machine and invoking application-level functions leads to an bytecode interpretation indirection. However, special care must be taken regarding exceptions because application level exceptions are wrapped into ``OperationErrors`` which are thus distinguished from plain interpreter-level exceptions. -See `application level exceptions`_ for some more information +See :ref:`application level exceptions <applevel-exceptions>` for some more information on ``OperationErrors``. The interpreter implementation offers mechanisms to allow a @@ -92,36 +90,29 @@ implemented via `interpreter descriptors`_ (also see Raymond Hettingers A significant complexity lies in `function argument parsing`_. Python as a language offers flexible ways of providing and receiving arguments for a particular function invocation. Not only does it take special care -to get this right, it also presents difficulties for the `annotation -pass`_ which performs a whole-program analysis on the +to get this right, it also presents difficulties for the :ref:`annotation +pass <rpython:annotator>` which performs a whole-program analysis on the bytecode interpreter, argument parsing and gatewaying code in order to infer the types of all values flowing across function calls. It is for this reason that PyPy resorts to generate -specialized frame classes and functions at `initialization -time`_ in order to let the annotator only see rather static +specialized frame classes and functions at :ref:`initialization +time <rpython:initialization-time>` in order to let the annotator only see rather static program flows with homogeneous name-value assignments on function invocations. -.. _`how-to guide for descriptors`: http://users.rcn.com/python/download/Descriptor.htm -.. _`annotation pass`: translation.html#the-annotation-pass -.. _`initialization time`: translation.html#initialization-time -.. _`interpreter-level and application-level`: coding-guide.html#interpreter-level -.. _`wrapped`: coding-guide.html#wrapping-rules -.. _`object space`: objspace.html -.. _`application level exceptions`: coding-guide.html#applevel-exceptions -.. _`here`: coding-guide.html#modules +.. _how-to guide for descriptors: http://users.rcn.com/python/download/Descriptor.htm Bytecode Interpreter Implementation Classes -================================================ +------------------------------------------- -.. _`Frame class`: -.. _`Frame`: +.. _Frame class: +.. _Frame: Frame classes ------------------ +~~~~~~~~~~~~~ The concept of Frames is pervasive in executing programs and on virtual machines in particular. They are sometimes called @@ -156,10 +147,11 @@ class are added in various files: - nested scope support is added to the ``PyFrame`` class in :source:`pypy/interpreter/nestedscope.py`. + .. _Code: Code Class ------------- +~~~~~~~~~~ PyPy's code objects contain the same information found in CPython's code objects. They differ from Function_ objects in that they are only immutable representations @@ -190,10 +182,11 @@ allow to create custom Frame objects extending the execution of functions in various ways. The several Frame_ classes already utilize this flexibility in order to implement Generators and Nested Scopes. + .. _Function: Function and Method classes ----------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~~ The PyPy ``Function`` class (in :source:`pypy/interpreter/function.py`) represents a Python function. A ``Function`` carries the following @@ -212,11 +205,13 @@ object holding a binding to an instance or a class. Finally, ``Functions`` and ``Methods`` both offer a ``call_args()`` method which executes the function given an `Arguments`_ class instance. + .. _Arguments: -.. _`function argument parsing`: + +.. _function argument parsing: Arguments Class --------------------- +~~~~~~~~~~~~~~~ The Argument class (in :source:`pypy/interpreter/argument.py`) is responsible for parsing arguments passed to functions. @@ -241,10 +236,10 @@ the bound object. The ``Arguments`` provides means to allow all this argument parsing and also cares for error reporting. -.. _`Module`: +.. _Module: Module Class -------------------- +~~~~~~~~~~~~ A ``Module`` instance represents execution state usually constructed from executing the module's source file. In addition to such a module's @@ -261,30 +256,25 @@ application-level files there is a more refined which allows to define name-value bindings both at application level and at interpreter level. See the ``__builtin__`` module's :source:`pypy/module/__builtin__/__init__.py` file for an -example and the higher level `chapter on Modules in the coding -guide`_. - -.. _`__builtin__ module`: https://bitbucket.org/pypy/pypy/src/tip/pypy/module/__builtin__/ -.. _`chapter on Modules in the coding guide`: coding-guide.html#modules +example and the higher level :ref:`chapter on Modules in the coding +guide <modules>`. -.. _`Gateway classes`: Gateway classes ----------------------- +~~~~~~~~~~~~~~~ A unique PyPy property is the ability to easily cross the barrier between interpreted and machine-level code (often referred to as -the difference between `interpreter-level and application-level`_). +the difference between :ref:`interpreter-level and application-level <interpreter-level>`). Be aware that the according code (in :source:`pypy/interpreter/gateway.py`) for crossing the barrier in both directions is somewhat involved, mostly due to the fact that the type-inferring annotator needs to keep track of the types of objects flowing across those barriers. -.. _typedefs: Making interpreter-level functions available at application-level -+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ In order to make an interpreter-level function available at application level, one invokes ``pypy.interpreter.gateway.interp2app(func)``. @@ -322,9 +312,9 @@ and hints to keep the type-inferring annotator happy. Calling into application level code from interpreter-level -+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ -Application level code is `often preferable`_. Therefore, +Application level code is :ref:`often preferable <app-preferable>`. Therefore, we often like to invoke application level code from interpreter-level. This is done via the Gateway's ``app2interp`` mechanism which we usually invoke at definition time in a module. @@ -378,11 +368,11 @@ Note that at a later point we can rewrite the ``find_metaclass`` implementation at interpreter-level and we would not have to modify the calling side at all. -.. _`often preferable`: coding-guide.html#app-preferable -.. _`interpreter descriptors`: + +.. _interpreter descriptors: Introspection and Descriptors ------------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Python traditionally has a very far-reaching introspection model for bytecode interpreter related objects. In PyPy and in CPython read diff --git a/pypy/doc/jit-hooks.rst b/pypy/doc/jit-hooks.rst index 449552a2ec..0879040451 100644 --- a/pypy/doc/jit-hooks.rst +++ b/pypy/doc/jit-hooks.rst @@ -1,5 +1,5 @@ -JIT hooks in PyPy -================= +JIT hooks +========= There are several hooks in the `pypyjit` module that may help you with understanding what's pypy's JIT doing while running your program. There diff --git a/pypy/doc/objspace-proxies.rst b/pypy/doc/objspace-proxies.rst index 4c404dfe7c..441aff09c4 100644 --- a/pypy/doc/objspace-proxies.rst +++ b/pypy/doc/objspace-proxies.rst @@ -1,11 +1,10 @@ -================================= What PyPy can do for your objects ================================= .. contents:: -Thanks to the `Object Space`_ architecture, any feature that is +Thanks to the :doc:`Object Space <objspace>` architecture, any feature that is based on proxying, extending, changing or otherwise controlling the behavior of all objects in a running program is easy to implement on top of PyPy. @@ -20,11 +19,11 @@ Here is what we have implemented so far, in historical order: control operations on application and builtin objects, e.g lists, dictionaries, tracebacks. -.. _`Object Space`: objspace.html + .. _tproxy: Transparent Proxies -================================ +------------------- PyPy's Transparent Proxies allow routing of operations on objects to a callable. Application level code can customize objects without @@ -35,8 +34,9 @@ giving you full control on all operations that are performed on the See [D12.1]_ for more context, motivation and usage of transparent proxies. + Example of the core mechanism -------------------------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The following example proxies a list and will return ``42`` on any add operation to the list:: @@ -54,11 +54,9 @@ return ``42`` on any add operation to the list:: >>>> i + 3 42 -.. _`alternative object implementations`: interpreter-optimizations.html - Example of recording all operations on builtins ----------------------------------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Suppose we want to have a list which stores all operations performed on it for later analysis. We can use the small :source:`lib_pypy/tputil.py` module to help @@ -88,10 +86,11 @@ Note that ``append`` shows up as ``__getattribute__`` and that ``type(lst)`` does not show up at all - the type is the only aspect of the instance which the controller cannot change. -.. _`transparent proxy builtins`: + +.. _transparent proxy builtins: Transparent Proxy PyPy builtins and support ------------------------------------------------------------ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ If you are using the `--objspace-std-withtproxy`_ option the `__pypy__`_ module provides the following builtins: @@ -106,12 +105,13 @@ the `__pypy__`_ module provides the following builtins: ``None`` is returned. .. _`__pypy__`: __pypy__-module.html -.. _`--objspace-std-withtproxy`: config/objspace.std.withtproxy.html +.. _--objspace-std-withtproxy: config/objspace.std.withtproxy.html + .. _tputil: tputil helper module ----------------------------- +~~~~~~~~~~~~~~~~~~~~ The :source:`lib_pypy/tputil.py` module provides: @@ -142,8 +142,9 @@ The :source:`lib_pypy/tputil.py` module provides: ``proxyoperation.delegate()`` to delegate the operation to this object instance. + Further points of interest ---------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~ A lot of tasks could be performed using transparent proxies, including, but not limited to: @@ -158,20 +159,21 @@ but not limited to: objects (of course some operations could raise exceptions, but since they are purely done on application level, that is not real problem) + Implementation Notes ------------------------------ +~~~~~~~~~~~~~~~~~~~~ PyPy's standard object space allows to internally have multiple implementations of a type and change the implementation at run time while application level code consistently sees the exact same type and object. Multiple performance optimizations using this features are already implemented: see the document -about `alternative object implementations`_. Transparent +about :doc:`alternative object implementations <interpreter-optimizations>`. Transparent Proxies use the architecture to provide control back to application level code. -Transparent proxies are implemented on top of the `standard object -space`_, in :source:`pypy/objspace/std/proxy_helpers.py`, :source:`pypy/objspace/std/proxyobject.py` and +Transparent proxies are implemented on top of the :ref:`standard object +space <standard-object-space>`, in :source:`pypy/objspace/std/proxy_helpers.py`, :source:`pypy/objspace/std/proxyobject.py` and :source:`pypy/objspace/std/transparent.py`. To use them you will need to pass a `--objspace-std-withtproxy`_ option to ``py.py`` or ``translate.py``. This registers implementations named @@ -182,7 +184,5 @@ implemented in the std objspace. The types of objects that can be proxied this way are user created classes & functions, lists, dicts, exceptions, tracebacks and frames. -.. _`standard object space`: objspace.html#the-standard-object-space - .. [D12.1] `High-Level Backends and Interpreter Feature Prototypes`, PyPy EU-Report, 2007, http://codespeak.net/pypy/extradoc/eu-report/D12.1_H-L-Backends_and_Feature_Prototypes-2007-03-22.pdf diff --git a/pypy/doc/objspace.rst b/pypy/doc/objspace.rst index 2388b9c717..54aa10e875 100644 --- a/pypy/doc/objspace.rst +++ b/pypy/doc/objspace.rst @@ -1,15 +1,14 @@ -================ The Object Space ================ .. contents:: -.. _`objectspace`: -.. _`Object Space`: +.. _objectspace: +.. _Object Space: Introduction -============ +------------ The object space creates all objects and knows how to perform operations on the objects. You may think of an object space as being a library @@ -19,7 +18,7 @@ operation is *add*: add's implementations are, for example, responsible for performing numeric addition when add works on numbers, concatenation when add works on built-in sequences. -All object-space operations take and return `application-level`_ objects. +All object-space operations take and return :ref:`application-level <application-level>` objects. There are only a few, very simple, object-space operations which allow the bytecode interpreter to gain some knowledge about the value of an application-level object. @@ -34,8 +33,8 @@ the bytecode interpreter: - The *Standard Object Space* is a complete implementation of the various built-in types and objects of Python. The Standard Object Space, together with the bytecode interpreter, is the foundation of our Python - implementation. Internally, it is a set of `interpreter-level`_ classes - implementing the various `application-level`_ objects -- integers, strings, + implementation. Internally, it is a set of :ref:`interpreter-level <interpreter-level>` classes + implementing the various :ref:`application-level <application-level>` objects -- integers, strings, lists, types, etc. To draw a comparison with CPython, the Standard Object Space provides the equivalent of the C structures ``PyIntObject``, ``PyListObject``, etc. @@ -48,26 +47,23 @@ the bytecode interpreter: - the *Flow Object Space* transforms a Python program into a flow-graph representation, by recording all operations that the bytecode interpreter would like to perform when it is shown the given Python - program. This technique is explained `in another document`_. + program. This technique is explained :doc:`in another document <rpython:translation>`. The present document gives a description of the above object spaces. The sources of PyPy contain the various object spaces in the directory :source:`pypy/objspace/`. -.. _`application-level`: coding-guide.html#application-level -.. _`interpreter-level`: coding-guide.html#interpreter-level -.. _`in another document`: translation.html -.. _interface: +.. _objspace-interface: Object Space Interface -====================== +---------------------- This is the public API that all Object Spaces implement. Administrative Functions ----------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~ ``getexecutioncontext():`` Return current active execution context @@ -77,8 +73,9 @@ Administrative Functions Return a Module object for the built-in module given by name (:source:`pypy/interpreter/module.py`). + Operations on Objects in the Object Space ------------------------------------------ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ These functions both take and return "wrapped" objects. @@ -118,8 +115,9 @@ semantic - they directly correspond to language-level constructs: ``exception_match(w_exc_type, w_check_class):`` Checks if the given exception type matches 'w_check_class'. Used in matching the actual exception raised with the list of those to catch in an except clause. (Returns a wrapped result too!) + Convenience Functions ---------------------- +~~~~~~~~~~~~~~~~~~~~~ The following functions are part of the object space interface but would not be strictly necessary because they can be expressed using several other object @@ -170,7 +168,7 @@ introduce them as shortcuts. Creation of Application Level objects ---------------------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ``wrap(x):`` Returns a wrapped object that is a reference to the interpreter-level object @@ -204,8 +202,9 @@ Creation of Application Level objects ``newunicode(codelist):`` Creates a unicode string from a list of integers. + Conversions from Application Level to Interpreter Level ----------------------------------------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ``unwrap(w_x):`` Return the Interpreter Level equivalent of w_x. DO NOT USE! @@ -251,7 +250,7 @@ Conversions from Application Level to Interpreter Level Data Members ------------------ +~~~~~~~~~~~~ + space.builtin: The Module containing the builtins + space.sys: The 'sys' Module @@ -283,13 +282,13 @@ Data Members non-wrapped objects). -.. _`standard-object-space`: +.. _standard-object-space: The Standard Object Space -========================= +------------------------- Introduction ------------- +~~~~~~~~~~~~ The Standard Object Space (:source:`pypy/objspace/std/`) is the direct equivalent of CPython's object library (the "Objects/" subdirectory in the distribution). It is an @@ -329,7 +328,7 @@ an integer (after all, integers are directly available in C too). You could represent small integers as odd-valuated pointers. But it puts extra burden on the whole C code, so the CPython team avoided it. (In our case it is an optimization that we eventually made, but not hard-coded at this level - -see `Standard Interpreter Optimizations`_.) +see :doc:`interpreter-optimizations`.) So in summary: wrapping integers as instances is the simple path, while using plain integers instead is the complex path, not the other way @@ -337,7 +336,7 @@ around. Object types ------------- +~~~~~~~~~~~~ The larger part of the :source:`pypy/objspace/std/` package defines and implements the library of Python's standard built-in object types. Each type (int, float, @@ -379,13 +378,11 @@ other implementations of strings use the same ``typedef`` from :source:`pypy/objspace/std/stringtype.py`. For other examples of multiple implementations of the same Python type, -see `Standard Interpreter Optimizations`_. - -.. _`Standard Interpreter Optimizations`: interpreter-optimizations.html +see :doc:`interpreter-optimizations`. Multimethods ------------- +~~~~~~~~~~~~ The Standard Object Space allows multiple object implementations per Python type - this is based on multimethods_. For a description of the @@ -444,7 +441,7 @@ overloading resolution mechanism (but occurs at runtime). Multimethod slicing -------------------- +~~~~~~~~~~~~~~~~~~~ The complete picture is more complicated because the Python object model is based on *descriptors*: the types ``int``, ``str``, etc. must have @@ -483,13 +480,14 @@ Additionally, slicing ensures that ``5 .__add__(6L)`` correctly returns ``add__Long_Long`` and there is no ``add__Int_Long``), which leads to ``6L.__radd__(5)`` being called, as in CPython. + .. _flow-object-space: The Flow Object Space -===================== +--------------------- Introduction ------------- +~~~~~~~~~~~~ The task of the FlowObjSpace (the source is at :source:`pypy/objspace/flow/`) is to generate a control-flow graph from a function. This graph will also contain a trace of the individual operations, so @@ -511,7 +509,7 @@ placeholder "wrapped objects" and give them to the interpreter, so that they appear in some next operation. This technique is an example of `Abstract Interpretation`_. -.. _`Abstract Interpretation`: http://en.wikipedia.org/wiki/Abstract_interpretation +.. _Abstract Interpretation: http://en.wikipedia.org/wiki/Abstract_interpretation For example, if the placeholder ``v1`` is given as the argument to the above function, the bytecode interpreter will call ``v2 = space.mul(space.wrap(3), @@ -523,18 +521,15 @@ result. During these calls the FlowObjSpace will record a basic block:: v3 = add(v2, Constant(2)) - The Flow model --------------- +~~~~~~~~~~~~~~ The data structures built up by the flow object space are described in the -`translation document`_. - -.. _`translation document`: translation.html#flow-model +:ref:`translation document <rpython:flow-model>`. How the FlowObjSpace works --------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~ The FlowObjSpace works by recording all operations issued by the bytecode interpreter into basic blocks. A basic block ends in one of two cases: when @@ -562,11 +557,9 @@ the bytecode interpreters calls ``is_true()``, or when a joinpoint is reached. Object Space proxies -==================== +-------------------- We have implemented several *proxy object spaces* which wrap another space (typically the standard one) and add some capability to all -objects. These object spaces are documented in a separate page: `What -PyPy can do for your objects`_. - -.. _`What PyPy can do for your objects`: objspace-proxies.html +objects. These object spaces are documented in a separate page: +:doc:`objspace-proxies` diff --git a/pypy/doc/parser.rst b/pypy/doc/parser.rst index 9e155046ad..d2a6bb54ed 100644 --- a/pypy/doc/parser.rst +++ b/pypy/doc/parser.rst @@ -1,13 +1,12 @@ - -=========== PyPy Parser =========== Overview -======== +-------- The PyPy parser includes a tokenizer and a recursive descent parser. + Tokenizer --------- @@ -15,13 +14,15 @@ At the moment, the tokenizer is implemented as a single function (``generate_tokens`` in :source:`pypy/interpreter/pyparser/pytokenizer.py`) that builds a list of tokens. The tokens are then fed to the parser. + Parser ------ The parser is a simple LL(1) parser that is similar to CPython's. + Building the Python grammar -*************************** +~~~~~~~~~~~~~~~~~~~~~~~~~~~ The python grammar is built at startup from the pristine CPython grammar file (see :source:`pypy/interpreter/pyparser/metaparser.py`). The grammar builder first @@ -34,8 +35,9 @@ parsing. Finally, the assigns the grammar builder assigns each DFA state a number and packs them into a list for the parser to use. The final product is an instance of the ``Grammar`` class in :source:`pypy/interpreter/pyparser/parser.py`. + Parser implementation -********************* +~~~~~~~~~~~~~~~~~~~~~ The workhorse of the parser is the ``add_token`` method of the ``Parser`` class. It tries to find a transition from the current state to another state based on @@ -44,8 +46,9 @@ if the current state is accepting. If it's not, a ``ParseError`` is raised. When parsing is done without error, the parser has built a tree of ``Node``. + Parsing Python -************** +~~~~~~~~~~~~~~ The glue code between the tokenizer and the parser as well as extra Python specific code is in :source:`pypy/interpreter/pyparser/pyparse.py`. The @@ -54,14 +57,16 @@ tree. It also detects the coding cookie if there is one and decodes the source. Note that __future__ imports are handled before the parser is invoked by manually parsing the source in :source:`pypy/interpreter/pyparser/future.py`. + Compiler -------- The next step in generating Python bytecode is converting the parse tree into an Abstract Syntax Tree (AST). + Building AST -************ +~~~~~~~~~~~~ Python's AST is described in :source:`pypy/interpreter/astcompiler/tools/Python.asdl`. From this definition, :source:`pypy/interpreter/astcompiler/tools/asdl_py.py` generates @@ -74,15 +79,17 @@ extensions to the AST classes are in parse trees into AST. It walks down the parse tree building nodes as it goes. The result is a toplevel ``mod`` node. + AST Optimization -**************** +~~~~~~~~~~~~~~~~ :source:`pypy/interpreter/astcompiler/optimize.py` contains the AST optimizer. It does constant folding of expressions, and other simple transformations like making a load of the name "None" into a constant. + Symbol analysis -*************** +~~~~~~~~~~~~~~~ Before writing bytecode, a symbol table is built in :source:`pypy/interpreter/astcompiler/symtable.py`. It determines if every name in the @@ -90,8 +97,9 @@ source is local, implicitly global (no global declaration), explicitly global (there's a global declaration of the name in the scope), a cell (the name in used in nested scopes), or free (it's used in a nested function). + Bytecode generation -******************* +~~~~~~~~~~~~~~~~~~~ Bytecode is emitted in :source:`pypy/interpreter/astcompiler/codegen.py`. Each bytecode is represented temporarily by the ``Instruction`` class in diff --git a/pypy/doc/project-documentation.rst b/pypy/doc/project-documentation.rst index ec726f07a7..53490ec72a 100644 --- a/pypy/doc/project-documentation.rst +++ b/pypy/doc/project-documentation.rst @@ -1,41 +1,53 @@ - Project Documentation -===================================== +===================== -`architecture`_ gives a complete view of PyPy's basic design. +:doc:`architecture <architecture>` gives a complete view of PyPy's basic design. -`coding guide`_ helps you to write code for PyPy (especially also describes +:doc:`coding guide <coding-guide>` helps you to write code for PyPy (especially also describes coding in RPython a bit). -`sprint reports`_ lists reports written at most of our sprints, from +:doc:`sprint reports <sprint-reports>` lists reports written at most of our sprints, from 2003 to the present. -`papers, talks and related projects`_ lists presentations +:doc:`papers, talks and related projects <extradoc>` lists presentations and related projects as well as our published papers. -`PyPy video documentation`_ is a page linking to the videos (e.g. of talks and +:doc:`PyPy video documentation <video-index>` is a page linking to the videos (e.g. of talks and introductions) that are available. -`Technical reports`_ is a page that contains links to the +:doc:`Technical reports <index-report>` is a page that contains links to the reports that we submitted to the European Union. -`development methodology`_ describes our sprint-driven approach. +:doc:`development methodology <dev_method>` describes our sprint-driven approach. -`LICENSE`_ contains licensing details (basically a straight MIT-license). +:source:`LICENSE` contains licensing details (basically a straight MIT-license). -`Glossary`_ of PyPy words to help you align your inner self with +:doc:`glossary` of PyPy words to help you align your inner self with the PyPy universe. +.. toctree:: + :hidden: + + architecture + coding-guide + sprint-reports + extradoc + video-index + index-report + dev_method + glossary + + Source Code Documentation -=============================================== +========================= -`object spaces`_ discusses the object space interface +:doc:`object spaces <objspace>` discusses the object space interface and several implementations. -`bytecode interpreter`_ explains the basic mechanisms +:doc:`bytecode interpreter <interpreter>` explains the basic mechanisms of the bytecode interpreter and virtual machine. -`interpreter optimizations`_ describes our various strategies for +:doc:`interpreter-optimizations` describes our various strategies for improving the performance of our interpreter, including alternative object implementations (for strings, dictionaries and lists) in the standard object space. @@ -43,97 +55,31 @@ standard object space. `dynamic-language translation`_ is a paper that describes the translation process, especially the flow object space and the annotator in detail. (This document is one -of the `EU reports`_.) +of the :doc:`EU reports <index-report>`.) -`low-level encapsulation`_ describes how our approach hides -away a lot of low level details. This document is also part -of the `EU reports`_. - -`translation aspects`_ describes how we weave different -properties into our interpreter during the translation -process. This document is also part of the `EU reports`_. - -`parser`_ contains (outdated, unfinished) documentation about +:doc:`parser <parser>` contains (outdated, unfinished) documentation about the parser. -`configuration documentation`_ describes the various configuration options that +:doc:`configuration documentation <config/index>` describes the various configuration options that allow you to customize PyPy. -`command line reference`_ - -`directory cross-reference`_ - -.. _`directory cross-reference`: dir-reference.html -.. _`command line reference`: commandline_ref.html -.. _`FAQ`: faq.html -.. _Glossary: glossary.html -.. _`PyPy video documentation`: video-index.html -.. _parser: parser.html -.. _`development methodology`: dev_method.html -.. _`sprint reports`: sprint-reports.html -.. _`papers, talks and related projects`: extradoc.html -.. _`object spaces`: objspace.html -.. _`interpreter optimizations`: interpreter-optimizations.html -.. _`dynamic-language translation`: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D05.1_Publish_on_translating_a_very-high-level_description.pdf -.. _`low-level encapsulation`: low-level-encapsulation.html -.. _`translation aspects`: translation-aspects.html -.. _`configuration documentation`: config/ -.. _`coding guide`: coding-guide.html -.. _`Architecture`: architecture.html -.. _`getting started`: getting-started.html -.. _`bytecode interpreter`: interpreter.html -.. _`EU reports`: index-report.html -.. _`Technical reports`: index-report.html -.. _`summary`: http://buildbot.pypy.org/summary -.. _`ideas for PyPy related projects`: project-ideas.html -.. _`Nightly builds and benchmarks`: http://tuatara.cs.uni-duesseldorf.de/benchmark.html -.. _`directory reference`: -.. _`Sandboxing Python code`: sandbox.html -.. _`LICENSE`: https://bitbucket.org/pypy/pypy/src/default/LICENSE - -.. The following documentation is important and reasonably up-to-date: - -.. extradoc: should this be integrated one level up: dcolish? - -.. toctree:: - :maxdepth: 1 - :hidden: +:doc:`command line reference <commandline_ref>` - interpreter.rst - objspace.rst - __pypy__-module.rst - objspace-proxies.rst - config/index.rst +:doc:`directory cross-reference <dir-reference>` - dev_method.rst - extending.rst +.. _dynamic-language translation: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D05.1_Publish_on_translating_a_very-high-level_description.pdf - extradoc.rst - video-index.rst - - glossary.rst - - contributor.rst - - interpreter-optimizations.rst - configuration.rst - parser.rst - - ctypes-implementation.rst - - how-to-release.rst - - index-report.rst - - stackless.rst - sandbox.rst - - discussions.rst +.. toctree:: + :hidden: - sprint-reports.rst + objspace + interpreter + interpreter-optimizations + parser + config/index + commandline_ref + dir-reference - eventhistory.rst - statistic/index.rst Indices and tables ================== diff --git a/pypy/doc/project-ideas.rst b/pypy/doc/project-ideas.rst index 1b92175ae2..e6907ccb9d 100644 --- a/pypy/doc/project-ideas.rst +++ b/pypy/doc/project-ideas.rst @@ -1,4 +1,3 @@ - Potential project list ====================== @@ -17,12 +16,17 @@ own improvement ideas. In any case, if you feel like working on some of those projects, or anything else in PyPy, pop up on IRC or write to us on the `mailing list`_. +.. _issue tracker: http://bugs.pypy.org +.. _mailing list: http://mail.python.org/mailman/listinfo/pypy-dev + + Make bytearray type fast ------------------------ PyPy's bytearray type is very inefficient. It would be an interesting task to look into possible optimizations on this. + Implement copy-on-write list slicing ------------------------------------ @@ -38,7 +42,8 @@ The numpy is rapidly progressing in pypy, so feel free to come to IRC and ask for proposed topic. A not necesarilly up-to-date `list of topics`_ is also available. -.. _`list of topics`: https://bitbucket.org/pypy/extradoc/src/extradoc/planning/micronumpy.txt +.. _list of topics: https://bitbucket.org/pypy/extradoc/src/extradoc/planning/micronumpy.txt + Improving the jitviewer ------------------------ @@ -64,25 +69,27 @@ the client): if you have great web developing skills and want to help PyPy, this is an ideal task to get started, because it does not require any deep knowledge of the internals. +.. _jitviewer: http://bitbucket.org/pypy/jitviewer + + Optimized Unicode Representation -------------------------------- -CPython 3.3 will use an `optimized unicode representation`_ which switches between +CPython 3.3 will use an optimized unicode representation (see :pep:`0393`) which switches between different ways to represent a unicode string, depending on whether the string fits into ASCII, has only two-byte characters or needs four-byte characters. The actual details would be rather differen in PyPy, but we would like to have the same optimization implemented. -.. _`optimized unicode representation`: http://www.python.org/dev/peps/pep-0393/ Translation Toolchain --------------------- * Incremental or distributed translation. - * Allow separate compilation of extension modules. + Various GCs ----------- @@ -92,14 +99,12 @@ experiments can be done for the general purpose. Examples: * An incremental garbage collector that has specified maximal pause times, crucial for games - * A garbage collector that compact memory better for mobile devices - * A concurrent garbage collector (a lot of work) - * A collector that keeps object flags in separate memory pages, to avoid un-sharing all pages between several fork()ed processes + STM (Software Transactional Memory) ----------------------------------- @@ -107,7 +112,7 @@ This is work in progress. Besides the main development path, whose goal is to make a (relatively fast) version of pypy which includes STM, there are independent topics that can already be experimented with on the existing, JIT-less pypy-stm version: - + * What kind of conflicts do we get in real use cases? And, sometimes, which data structures would be more appropriate? For example, a dict implemented as a hash table will suffer "stm collisions" in all threads @@ -115,14 +120,12 @@ JIT-less pypy-stm version: implementations. Maybe alternate strategies can be implemented at the level of the Python interpreter (see list/dict strategies, ``pypy/objspace/std/{list,dict}object.py``). - * More generally, there is the idea that we would need some kind of "debugger"-like tool to "debug" things that are not bugs, but stm conflicts. How would this tool look like to the end Python programmers? Like a profiler? Or like a debugger with breakpoints on aborted transactions? It would probably be all app-level, with a few hooks e.g. for transaction conflicts. - * Find good ways to have libraries using internally threads and atomics, but not exposing threads to the user. Right now there is a rough draft in ``lib_pypy/transaction.py``, but much better is possible. For example @@ -141,6 +144,7 @@ to be got from them!): * `hg` + Experiment (again) with LLVM backend for RPython compilation ------------------------------------------------------------ @@ -155,6 +159,7 @@ better project to try to optimize the alternative, the "shadowstack" GC root finder, which is nicely portable. So far it gives a pypy that is around 7% slower.) + Embedding PyPy ---------------------------------------- @@ -166,6 +171,7 @@ it runs a .py module that installs (via ctypes) the interface it wants exported. This would give us a one-size-fits-all generic .so file to be imported by any application that wants to load .so files :-) + Optimising cpyext (CPython C-API compatibility layer) ----------------------------------------------------- @@ -182,7 +188,3 @@ have CPython extensions run much faster on PyPy than they currently do, or to make them work at all if they currently don't. A part of this work would be to get cpyext into a shape where it supports running Cython generated extensions. - -.. _`issue tracker`: http://bugs.pypy.org -.. _`mailing list`: http://mail.python.org/mailman/listinfo/pypy-dev -.. _`jitviewer`: http://bitbucket.org/pypy/jitviewer diff --git a/pypy/doc/releases/0.6.rst b/pypy/doc/releases/0.6.rst index 0f32cc3edf..df3ad804ee 100644 --- a/pypy/doc/releases/0.6.rst +++ b/pypy/doc/releases/0.6.rst @@ -1,12 +1,12 @@ The PyPy 0.6 release ==================== -*The PyPy Development Team is happy to announce the first +*The PyPy Development Team is happy to announce the first public release of PyPy after two years of spare-time and -half a year of EU funded development. The 0.6 release -is eminently a preview release.* +half a year of EU funded development. The 0.6 release +is eminently a preview release.* -What it is and where to start +What it is and where to start ----------------------------- Getting started: getting-started.html @@ -29,7 +29,7 @@ regression tests that do not depend deeply on C-extensions. Some of that functionality is still made available by PyPy piggy-backing on the host CPython interpreter. Double interpretation and abstractions in the code-base make it so that PyPy running on CPython is quite slow -(around 2000x slower than CPython ), this is expected. +(around 2000x slower than CPython), this is expected. This release is intended for people that want to look and get a feel into what we are doing, playing with interpreter and perusing the @@ -38,8 +38,8 @@ codebase. Possibly to join in the fun and efforts. Interesting bits and highlights --------------------------------- -The release is also a snap-shot of our ongoing efforts towards -low-level translation and experimenting with unique features. +The release is also a snap-shot of our ongoing efforts towards +low-level translation and experimenting with unique features. * By default, PyPy is a Python version that works completely with new-style-classes semantics. However, support for old-style classes @@ -47,9 +47,9 @@ low-level translation and experimenting with unique features. their metaclass and instance object are included and can be re-made the default with the ``--oldstyle`` option. -* In PyPy, bytecode interpretation and object manipulations - are well separated between a bytecode interpreter and an - *object space* which implements operations on objects. +* In PyPy, bytecode interpretation and object manipulations + are well separated between a bytecode interpreter and an + *object space* which implements operations on objects. PyPy comes with experimental object spaces augmenting the standard one through delegation: @@ -58,17 +58,17 @@ low-level translation and experimenting with unique features. * the 'thunk' object space that implements lazy values and a 'become' operation that can exchange object identities. - + These spaces already give a glimpse in the flexibility potential of PyPy. See demo/fibonacci.py and demo/sharedref.py for examples about the 'thunk' object space. -* The 0.6 release also contains a snapshot of our translation-efforts +* The 0.6 release also contains a snapshot of our translation-efforts to lower level languages. For that we have developed an annotator which is capable of inferring type information across our code base. The annotator right now is already capable of successfully type annotating basically *all* of - PyPy code-base, and is included with 0.6. + PyPy code-base, and is included with 0.6. * From type annotated code, low-level code needs to be generated. Backends for various targets (C, LLVM,...) are included; they are @@ -80,35 +80,35 @@ Ongoing work and near term goals --------------------------------- Generating low-level code is the main area we are hammering on in the -next months; our plan is to produce a PyPy version in August/September -that does not need to be interpreted by CPython anymore and will -thus run considerably faster than the 0.6 preview release. +next months; our plan is to produce a PyPy version in August/September +that does not need to be interpreted by CPython anymore and will +thus run considerably faster than the 0.6 preview release. PyPy has been a community effort from the start and it would not have got that far without the coding and feedback support -from numerous people. Please feel free to give feedback and -raise questions. +from numerous people. Please feel free to give feedback and +raise questions. contact points: http://pypy.org/contact.html contributor list: contributor.html -have fun, +have fun, + + Armin Rigo, Samuele Pedroni, - Armin Rigo, Samuele Pedroni, + Holger Krekel, Christian Tismer, - Holger Krekel, Christian Tismer, + Carl Friedrich Bolz - Carl Friedrich Bolz + PyPy development and activities happen as an open source project + and with the support of a consortium funded by a two year EU IST + research grant. Here is a list of partners of the EU project: - PyPy development and activities happen as an open source project - and with the support of a consortium funded by a two year EU IST - research grant. Here is a list of partners of the EU project: - Heinrich-Heine University (Germany), AB Strakt (Sweden) - merlinux GmbH (Germany), tismerysoft GmbH(Germany) + merlinux GmbH (Germany), tismerysoft GmbH(Germany) Logilab Paris (France), DFKI GmbH (Germany) diff --git a/pypy/doc/releases/0.7.0.rst b/pypy/doc/releases/0.7.0.rst index b1b33b0bb1..9e33cd50ab 100644 --- a/pypy/doc/releases/0.7.0.rst +++ b/pypy/doc/releases/0.7.0.rst @@ -1,28 +1,28 @@ pypy-0.7.0: first PyPy-generated Python Implementations -============================================================== +======================================================= -What was once just an idea between a few people discussing -on some nested mailing list thread and in a pub became reality ... +What was once just an idea between a few people discussing +on some nested mailing list thread and in a pub became reality ... the PyPy development team is happy to announce its first public release of a fully translatable self contained Python implementation. The 0.7 release showcases the results of our efforts in the last few months since the 0.6 preview release which have been partially funded by the European Union: -- whole program type inference on our Python Interpreter - implementation with full translation to two different - machine-level targets: C and LLVM +- whole program type inference on our Python Interpreter + implementation with full translation to two different + machine-level targets: C and LLVM -- a translation choice of using a refcounting or Boehm +- a translation choice of using a refcounting or Boehm garbage collectors -- the ability to translate with or without thread support +- the ability to translate with or without thread support -- very complete language-level compliance with CPython 2.4.1 +- very complete language-level compliance with CPython 2.4.1 -What is PyPy (about)? ------------------------------------------------- +What is PyPy (about)? +--------------------- PyPy is a MIT-licensed research-oriented reimplementation of Python written in Python itself, flexible and easy to @@ -39,23 +39,23 @@ aspect - should become robust against language changes. Note that PyPy is mainly a research and development project and does not by itself focus on getting a production-ready Python implementation although we do hope and expect it to -become a viable contender in that area sometime next year. +become a viable contender in that area sometime next year. -Where to start? ------------------------------ +Where to start? +--------------- Getting started: http://codespeak.net/pypy/dist/pypy/doc/getting-started.html -PyPy Documentation: http://codespeak.net/pypy/dist/pypy/doc/ +PyPy Documentation: http://codespeak.net/pypy/dist/pypy/doc/ PyPy Homepage: http://codespeak.net/pypy/ The interpreter and object model implementations shipped with the 0.7 version can run on their own and implement the core language features of Python as of CPython 2.4. However, we still -do not recommend using PyPy for anything else than for education, -playing or research purposes. +do not recommend using PyPy for anything else than for education, +playing or research purposes. Ongoing work and near term goals --------------------------------- @@ -63,42 +63,42 @@ Ongoing work and near term goals PyPy has been developed during approximately 15 coding sprints across Europe and the US. It continues to be a very dynamically and incrementally evolving project with many -one-week meetings to follow. You are invited to consider coming to -the next such meeting in Paris mid October 2005 where we intend to +one-week meetings to follow. You are invited to consider coming to +the next such meeting in Paris mid October 2005 where we intend to plan and head for an even more intense phase of the project involving building a JIT-Compiler and enabling unique features not found in other Python language implementations. PyPy has been a community effort from the start and it would not have got that far without the coding and feedback support -from numerous people. Please feel free to give feedback and -raise questions. +from numerous people. Please feel free to give feedback and +raise questions. contact points: http://codespeak.net/pypy/dist/pypy/doc/contact.html contributor list: http://codespeak.net/pypy/dist/pypy/doc/contributor.html -have fun, - +have fun, + the pypy team, of which here is a partial snapshot - of mainly involved persons: - - Armin Rigo, Samuele Pedroni, - Holger Krekel, Christian Tismer, - Carl Friedrich Bolz, Michael Hudson, - Eric van Riet Paap, Richard Emslie, - Anders Chrigstroem, Anders Lehmann, - Ludovic Aubry, Adrien Di Mascio, - Niklaus Haldimann, Jacob Hallen, - Bea During, Laura Creighton, - and many contributors ... - -PyPy development and activities happen as an open source project -and with the support of a consortium partially funded by a two -year European Union IST research grant. Here is a list of -the full partners of that consortium: - + of mainly involved persons: + + Armin Rigo, Samuele Pedroni, + Holger Krekel, Christian Tismer, + Carl Friedrich Bolz, Michael Hudson, + Eric van Riet Paap, Richard Emslie, + Anders Chrigstroem, Anders Lehmann, + Ludovic Aubry, Adrien Di Mascio, + Niklaus Haldimann, Jacob Hallen, + Bea During, Laura Creighton, + and many contributors ... + +PyPy development and activities happen as an open source project +and with the support of a consortium partially funded by a two +year European Union IST research grant. Here is a list of +the full partners of that consortium: + Heinrich-Heine University (Germany), AB Strakt (Sweden) - merlinux GmbH (Germany), tismerysoft GmbH(Germany) + merlinux GmbH (Germany), tismerysoft GmbH(Germany) Logilab Paris (France), DFKI GmbH (Germany) ChangeMaker (Sweden), Impara (Germany) diff --git a/pypy/doc/releases/0.8.0.rst b/pypy/doc/releases/0.8.0.rst index 19b90ae1b9..ad851622e0 100644 --- a/pypy/doc/releases/0.8.0.rst +++ b/pypy/doc/releases/0.8.0.rst @@ -1,24 +1,24 @@ -pypy-0.8.0: Translatable compiler/parser and some more speed +pypy-0.8.0: Translatable compiler/parser and some more speed ============================================================== -The PyPy development team has been busy working and we've now packaged -our latest improvements, completed work and new experiments as +The PyPy development team has been busy working and we've now packaged +our latest improvements, completed work and new experiments as version 0.8.0, our third public release. The highlights of this third release of PyPy are: - Translatable parser and AST compiler. PyPy now integrates its own compiler based on Python own 'compiler' package but with a number - of fixes and code simplifications in order to get it translated - with the rest of PyPy. This makes using the translated pypy - interactively much more pleasant, as compilation is considerably + of fixes and code simplifications in order to get it translated + with the rest of PyPy. This makes using the translated pypy + interactively much more pleasant, as compilation is considerably faster than in 0.7.0. - Some Speed enhancements. Translated PyPy is now about 10 times faster than 0.7 but still 10-20 times slower than - CPython on pystones and other benchmarks. At the same time, + CPython on pystones and other benchmarks. At the same time, language compliance has been slightly increased compared to 0.7 - which had already reached major CPython compliance goals. + which had already reached major CPython compliance goals. - Some experimental features are now translatable. Since 0.6.0, PyPy shipped with an experimental Object Space (the part of PyPy @@ -27,7 +27,7 @@ The highlights of this third release of PyPy are: object space can also be translated preserving its feature additions. -What is PyPy (about)? +What is PyPy (about)? ------------------------------------------------ PyPy is a MIT-licensed research-oriented reimplementation of @@ -45,25 +45,25 @@ aspect - should become robust against language changes. Note that PyPy is mainly a research and development project and does not by itself focus on getting a production-ready Python implementation although we do hope and expect it to -become a viable contender in that area sometime next year. +become a viable contender in that area sometime next year. -PyPy is partially funded as a research project under the -European Union's IST programme. +PyPy is partially funded as a research project under the +European Union's IST programme. -Where to start? +Where to start? ----------------------------- Getting started: http://codespeak.net/pypy/dist/pypy/doc/getting-started.html -PyPy Documentation: http://codespeak.net/pypy/dist/pypy/doc/ +PyPy Documentation: http://codespeak.net/pypy/dist/pypy/doc/ PyPy Homepage: http://codespeak.net/pypy/ The interpreter and object model implementations shipped with the 0.8 version can run on their own and implement the core language features of Python as of CPython 2.4. However, we still -do not recommend using PyPy for anything else than for education, -playing or research purposes. +do not recommend using PyPy for anything else than for education, +playing or research purposes. Ongoing work and near term goals --------------------------------- @@ -74,11 +74,11 @@ started to scratch the surface of Just-In-Time compiler related work, which we still expect will be the major source of our future speed improvements and some successful amount of work has been done on the support needed for stackless-like features. - + This release also includes the snapshots in preliminary or embryonic form of the following interesting but yet not completed sub projects: -- The OOtyper, a RTyper variation for higher-level backends +- The OOtyper, a RTyper variation for higher-level backends (Squeak, ...) - A JavaScript backend - A limited (PPC) assembler backend (this related to the JIT) @@ -91,25 +91,25 @@ to follow. PyPy has been a community effort from the start and it would not have got that far without the coding and feedback support -from numerous people. Please feel free to give feedback and -raise questions. +from numerous people. Please feel free to give feedback and +raise questions. contact points: http://codespeak.net/pypy/dist/pypy/doc/contact.html -have fun, - - the pypy team, (Armin Rigo, Samuele Pedroni, - Holger Krekel, Christian Tismer, - Carl Friedrich Bolz, Michael Hudson, +have fun, + + the pypy team, (Armin Rigo, Samuele Pedroni, + Holger Krekel, Christian Tismer, + Carl Friedrich Bolz, Michael Hudson, and many others: http://codespeak.net/pypy/dist/pypy/doc/contributor.html) -PyPy development and activities happen as an open source project -and with the support of a consortium partially funded by a two -year European Union IST research grant. The full partners of that -consortium are: - +PyPy development and activities happen as an open source project +and with the support of a consortium partially funded by a two +year European Union IST research grant. The full partners of that +consortium are: + Heinrich-Heine University (Germany), AB Strakt (Sweden) - merlinux GmbH (Germany), tismerysoft GmbH (Germany) + merlinux GmbH (Germany), tismerysoft GmbH (Germany) Logilab Paris (France), DFKI GmbH (Germany) ChangeMaker (Sweden), Impara (Germany) diff --git a/pypy/doc/releases/0.9.0.rst b/pypy/doc/releases/0.9.0.rst index 620b1bf398..6f6e4f38b5 100644 --- a/pypy/doc/releases/0.9.0.rst +++ b/pypy/doc/releases/0.9.0.rst @@ -1,8 +1,8 @@ pypy-0.9.0: stackless, new extension compiler ============================================================== -The PyPy development team has been busy working and we've now packaged -our latest improvements, completed work and new experiments as +The PyPy development team has been busy working and we've now packaged +our latest improvements, completed work and new experiments as version 0.9.0, our fourth public release. The highlights of this fourth release of PyPy are: @@ -28,7 +28,7 @@ The highlights of this fourth release of PyPy are: standard-in-Python-2.5 ctypes module. See its documentation for more: http://codespeak.net/pypy/dist/pypy/doc/rctypes.html -**framework GCs** +**framework GCs** PyPy's interpreter can now be compiled to use a garbage collector written in RPython. This added control over PyPy's execution makes the implementation of new and interesting features possible, apart from @@ -61,7 +61,7 @@ The highlights of this fourth release of PyPy are: We now run all our tests every night, and you can see the summary at: http://buildbot.pypy.org/summary -What is PyPy (about)? +What is PyPy (about)? ------------------------------------------------ PyPy is a MIT-licensed research-oriented reimplementation of Python @@ -83,12 +83,12 @@ contender in that area sometime next year. PyPy is partially funded as a research project under the European Union's IST programme. -Where to start? +Where to start? ----------------------------- Getting started: http://codespeak.net/pypy/dist/pypy/doc/getting-started.html -PyPy Documentation: http://codespeak.net/pypy/dist/pypy/doc/ +PyPy Documentation: http://codespeak.net/pypy/dist/pypy/doc/ PyPy Homepage: http://codespeak.net/pypy/ @@ -115,24 +115,24 @@ to follow. PyPy has been a community effort from the start and it would not have got that far without the coding and feedback support -from numerous people. Please feel free to give feedback and -raise questions. +from numerous people. Please feel free to give feedback and +raise questions. contact points: http://codespeak.net/pypy/dist/pypy/doc/contact.html -have fun, - - the pypy team, (Armin Rigo, Samuele Pedroni, - Holger Krekel, Christian Tismer, - Carl Friedrich Bolz, Michael Hudson, +have fun, + + the pypy team, (Armin Rigo, Samuele Pedroni, + Holger Krekel, Christian Tismer, + Carl Friedrich Bolz, Michael Hudson, and many others: http://codespeak.net/pypy/dist/pypy/doc/contributor.html) -PyPy development and activities happen as an open source project -and with the support of a consortium partially funded by a two -year European Union IST research grant. The full partners of that -consortium are: - +PyPy development and activities happen as an open source project +and with the support of a consortium partially funded by a two +year European Union IST research grant. The full partners of that +consortium are: + Heinrich-Heine University (Germany), AB Strakt (Sweden) - merlinux GmbH (Germany), tismerysoft GmbH (Germany) + merlinux GmbH (Germany), tismerysoft GmbH (Germany) Logilab Paris (France), DFKI GmbH (Germany) ChangeMaker (Sweden), Impara (Germany) diff --git a/pypy/doc/releases/0.99.0.rst b/pypy/doc/releases/0.99.0.rst index a3b454f0b0..655fdeca9f 100644 --- a/pypy/doc/releases/0.99.0.rst +++ b/pypy/doc/releases/0.99.0.rst @@ -1,37 +1,37 @@ ====================================================================== -pypy-0.99.0: new object spaces, optimizations, configuration ... +pypy-0.99.0: new object spaces, optimizations, configuration ... ====================================================================== Welcome to the PyPy 0.99.0 release - a major snapshot -and milestone of the last 8 months of work and contributions +and milestone of the last 8 months of work and contributions since PyPy-0.9.0 came out in June 2006! -Main entry point for getting-started/download and documentation: +Main entry point for getting-started/download and documentation: http://codespeak.net/pypy/dist/pypy/doc/index.html Further below you'll find some notes about PyPy, -the 0.99.0 highlights and our aims for PyPy 1.0. +the 0.99.0 highlights and our aims for PyPy 1.0. -have fun, +have fun, - the PyPy team, + the PyPy team, Samuele Pedroni, Carl Friedrich Bolz, Armin Rigo, Michael Hudson, Maciej Fijalkowski, Anders Chrigstroem, Holger Krekel, Guido Wesdorp - and many others: + and many others: http://codespeak.net/pypy/dist/pypy/doc/contributor.html -What is PyPy? +What is PyPy? ================================ -Technically, PyPy is both a Python Interpreter implementation -and an advanced Compiler, actually a framework for implementing +Technically, PyPy is both a Python Interpreter implementation +and an advanced Compiler, actually a framework for implementing dynamic languages and generating virtual machines for them. The Framework allows for alternative frontends and -for alternative backends, currently C, LLVM and .NET. +for alternative backends, currently C, LLVM and .NET. For our main target "C", we can can "mix in" different Garbage Collectors and threading models, including micro-threads aka "Stackless". The inherent complexity that arises from this @@ -41,36 +41,36 @@ interpreter implementation, our main frontend. Socially, PyPy is a collaborative effort of many individuals working together in a distributed and sprint-driven way since 2003. PyPy would not have gotten as far without the coding, -feedback and general support from numerous people. +feedback and general support from numerous people. Formally, many of the current developers are involved in executing an EU contract with the goal of exploring and researching new approaches to Language/Compiler development and software engineering. This contract's duration is about to end March 2007 and we are working and preparing the according -final review which is scheduled for May 2007. +final review which is scheduled for May 2007. -Key 0.99.0 Features +Key 0.99.0 Features ===================== * new object spaces: - - Tainting: a 270-line proxy object space tracking - and boxing sensitive information within an application. - A tainted object is completely barred from crossing + - Tainting: a 270-line proxy object space tracking + and boxing sensitive information within an application. + A tainted object is completely barred from crossing an I/O barrier, such as writing to files, databases - or sockets. This allows to significantly reduce the - effort of e.g. security reviews to the few places where - objects are "declassified" in order to send information - across I/O barriers. + or sockets. This allows to significantly reduce the + effort of e.g. security reviews to the few places where + objects are "declassified" in order to send information + across I/O barriers. - Transparent proxies: allow to customize both application and builtin objects from application level code. Works as an addition to the Standard Object Space (and is translatable). For details see http://codespeak.net/pypy/dist/pypy/doc/proxy.html - -* optimizations: + +* optimizations: - Experimental new optimized implementations for various built in Python types (strings, dicts, lists) @@ -81,7 +81,7 @@ Key 0.99.0 Features - Improved inlining (now also working for higher level backends) and malloc removal. - - twice the speed of the 0.9 release, overall 2-3 slower than CPython + - twice the speed of the 0.9 release, overall 2-3 slower than CPython * High level backends: @@ -92,15 +92,15 @@ Key 0.99.0 Features - the JavaScript backend has evolved to a point where it can be used to write AJAX web applications with it. This is still an experimental technique, though. For demo applications see: - http://play1.codespeak.net/ + http://play1.codespeak.net/ -* new configuration system: - There is a new comprehensive configuration system that allows +* new configuration system: + There is a new comprehensive configuration system that allows fine-grained configuration of the PyPy standard interpreter and the - translation process. + translation process. * new and improved modules: Since the last release, the signal, mmap, bz2 - and fcntl standard library modules have been implemented for PyPy. The socket, + and fcntl standard library modules have been implemented for PyPy. The socket, _sre and os modules have been greatly improved. In addition we added a the pypymagic module that contains PyPy-specific functionality. @@ -116,60 +116,60 @@ Key 0.99.0 Features http://codespeak.net/pypy/dist/pypy/doc/rlib.html * improved documentation: - + - extended documentation about stackless features: http://codespeak.net/pypy/dist/pypy/doc/stackless.html - + - PyPy video documentation: eight hours of talks, interviews and features: http://codespeak.net/pypy/dist/pypy/doc/video-index.html - technical reports about various aspects of PyPy: http://codespeak.net/pypy/dist/pypy/doc/index-report.html - + The entry point to all our documentation is: http://codespeak.net/pypy/dist/pypy/doc/index.html -What about 1.0? +What about 1.0? ====================== In the last week leading up to the release, we decided to go for tagging the release as 0.99.0, mainly because -we have some efforts pending to integrate and complete -research and coding work: +we have some efforts pending to integrate and complete +research and coding work: * the JIT Compiler Generator is ready, but not fully integrated with the PyPy interpreter. As a result, the JIT does not give actual speed improvements yet, so we chose to leave it out of the 0.99 release: the result doesn't meet yet the speed expectations - that we set for ourselves - and which some blogs and people + that we set for ourselves - and which some blogs and people have chosen as the main criterium for looking at PyPy. * the extension enabling runtime changes of the Python grammar is not yet integrated. This will be used to provide Aspect-Oriented - Programming extensions and Design by Contract facilities in PyPy. + Programming extensions and Design by Contract facilities in PyPy. * the Logic object space, which provides Logic Variables in PyPy, needs to undergo a bit more testing. A constraint problem solver - extension module is ready, and needs to be integrated with the codebase. + extension module is ready, and needs to be integrated with the codebase. PyPy 0.99 is the start for getting to 1.0 end of March 2007, -which we intend to become a base for a longer (and more relaxed :) -time to come. +which we intend to become a base for a longer (and more relaxed :) +time to come. Funding partners and organizations ===================================================== - -PyPy development and activities happen as an open source project + +PyPy development and activities happen as an open source project and with the support of a consortium partially funded by a 28 months -European Union IST research grant. The full partners of that -consortium are: - +European Union IST research grant. The full partners of that +consortium are: + Heinrich-Heine University (Germany), Open End (Sweden) - merlinux GmbH (Germany), tismerysoft GmbH (Germany) + merlinux GmbH (Germany), tismerysoft GmbH (Germany) Logilab Paris (France), DFKI GmbH (Germany) ChangeMaker (Sweden), Impara (Germany) diff --git a/pypy/doc/releases/1.0.0.rst b/pypy/doc/releases/1.0.0.rst index 4bfcf84116..08580acbd6 100644 --- a/pypy/doc/releases/1.0.0.rst +++ b/pypy/doc/releases/1.0.0.rst @@ -2,8 +2,8 @@ PyPy 1.0: JIT compilers for free and more ========================================== -Welcome to the PyPy 1.0 release - a milestone integrating the results -of four years of research, engineering, management and sprinting +Welcome to the PyPy 1.0 release - a milestone integrating the results +of four years of research, engineering, management and sprinting efforts, concluding the 28 months phase of EU co-funding! Although still not mature enough for general use, PyPy 1.0 materializes diff --git a/pypy/doc/releases/1.1.0.rst b/pypy/doc/releases/1.1.0.rst index f7d1502e65..4b11004fe2 100644 --- a/pypy/doc/releases/1.1.0.rst +++ b/pypy/doc/releases/1.1.0.rst @@ -8,7 +8,7 @@ compatible with CPython (currently CPython 2.5) and on making the interpreter more stable and bug-free. Download page: - + http://codespeak.net/pypy/dist/pypy/doc/download.html PyPy's Getting Started lives at: @@ -19,7 +19,7 @@ Highlights of This Release ========================== - More of CPython's standard library extension modules are supported, - among them ctypes, sqlite3, csv, and many more. Most of these extension + among them ctypes, sqlite3, csv, and many more. Most of these extension modules are fully supported under Windows as well. http://codespeak.net/pypy/dist/pypy/doc/cpython_differences.html @@ -51,7 +51,7 @@ Highlights of This Release - Sandboxing support: It is possible to translate the Python interpreter in a special way so that the result is fully sandboxed. - + http://codespeak.net/pypy/dist/pypy/doc/sandbox.html http://blog.sandbox.lt/en/WSGI%20and%20PyPy%20sandbox @@ -137,7 +137,7 @@ general support from numerous people. Have fun, the PyPy release team, [in alphabetical order] - + Amaury Forgeot d'Arc, Anders Hammerquist, Antonio Cuni, Armin Rigo, Carl Friedrich Bolz, Christian Tismer, Holger Krekel, Maciek Fijalkowski, Samuele Pedroni diff --git a/pypy/doc/releases/1.4.0.rst b/pypy/doc/releases/1.4.0.rst index 13812d7650..87d486584c 100644 --- a/pypy/doc/releases/1.4.0.rst +++ b/pypy/doc/releases/1.4.0.rst @@ -47,11 +47,11 @@ More highlights * Other speed improvements, like JITted calls to functions like map(). .. _virtualenv: http://pypi.python.org/pypi/virtualenv -.. _`Virtualenv support`: http://morepypy.blogspot.com/2010/08/using-virtualenv-with-pypy.html -.. _`in production`: http://morepypy.blogspot.com/2010/11/running-large-radio-telescope-software.html -.. _`our blog`: http://morepypy.blogspot.com -.. _`pypy 1.4 and pypy 1.3`: http://speed.pypy.org/comparison/?exe=1%2B41,1%2B172&ben=1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20&env=1&hor=false&bas=1%2B41&chart=normal+bars -.. _`pypy 1.4 and cpython 2.6`: http://speed.pypy.org/comparison/?exe=2%2B35,1%2B172&ben=1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20&env=1&hor=false&bas=2%2B35&chart=normal+bars +.. _Virtualenv support: http://morepypy.blogspot.com/2010/08/using-virtualenv-with-pypy.html +.. _in production: http://morepypy.blogspot.com/2010/11/running-large-radio-telescope-software.html +.. _our blog: http://morepypy.blogspot.com +.. _pypy 1.4 and pypy 1.3: http://speed.pypy.org/comparison/?exe=1%2B41,1%2B172&ben=1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20&env=1&hor=false&bas=1%2B41&chart=normal+bars +.. _pypy 1.4 and cpython 2.6: http://speed.pypy.org/comparison/?exe=2%2B35,1%2B172&ben=1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20&env=1&hor=false&bas=2%2B35&chart=normal+bars Cheers, diff --git a/pypy/doc/releases/1.4.0beta.rst b/pypy/doc/releases/1.4.0beta.rst index d0bcd318f5..c52517dc8a 100644 --- a/pypy/doc/releases/1.4.0beta.rst +++ b/pypy/doc/releases/1.4.0beta.rst @@ -33,4 +33,4 @@ Highlights: Cheers, The PyPy team -.. _`list of patches`: https://bitbucket.org/pypy/pypy/src/tip/pypy/module/cpyext/patches/ +.. _list of patches: https://bitbucket.org/pypy/pypy/src/tip/pypy/module/cpyext/patches/ diff --git a/pypy/doc/releases/1.5.0.rst b/pypy/doc/releases/1.5.0.rst index 92da88e276..8de7944194 100644 --- a/pypy/doc/releases/1.5.0.rst +++ b/pypy/doc/releases/1.5.0.rst @@ -59,15 +59,15 @@ Carl Friedrich Bolz, Laura Creighton, Antonio Cuni, Maciej Fijalkowski, Amaury Forgeot d'Arc, Alex Gaynor, Armin Rigo and the PyPy team -.. _`CPython 2.6`: http://docs.python.org/dev/whatsnew/2.6.html -.. _`CPython 2.7`: http://docs.python.org/dev/whatsnew/2.7.html - -.. _`our blog`: http://morepypy.blogspot.com -.. _`pypy 1.5 and pypy 1.4`: http://bit.ly/joPhHo -.. _`pypy 1.5 and cpython 2.6.2`: http://bit.ly/mbVWwJ - -.. _`loop invariant code motion`: http://morepypy.blogspot.com/2011/01/loop-invariant-code-motion.html -.. _`compatibility wiki`: https://bitbucket.org/pypy/compatibility/wiki/Home -.. _`Tkinter and IDLE`: http://morepypy.blogspot.com/2011/04/using-tkinter-and-idle-with-pypy.html -.. _`cProfile`: http://docs.python.org/library/profile.html -.. _`external fork`: https://bitbucket.org/alex_gaynor/pypy-postgresql +.. _CPython 2.6: http://docs.python.org/dev/whatsnew/2.6.html +.. _CPython 2.7: http://docs.python.org/dev/whatsnew/2.7.html + +.. _our blog: http://morepypy.blogspot.com +.. _pypy 1.5 and pypy 1.4: http://bit.ly/joPhHo +.. _pypy 1.5 and cpython 2.6.2: http://bit.ly/mbVWwJ + +.. _loop invariant code motion: http://morepypy.blogspot.com/2011/01/loop-invariant-code-motion.html +.. _compatibility wiki: https://bitbucket.org/pypy/compatibility/wiki/Home +.. _Tkinter and IDLE: http://morepypy.blogspot.com/2011/04/using-tkinter-and-idle-with-pypy.html +.. _cProfile: http://docs.python.org/library/profile.html +.. _external fork: https://bitbucket.org/alex_gaynor/pypy-postgresql diff --git a/pypy/doc/releases/1.6.0.rst b/pypy/doc/releases/1.6.0.rst index edbfec30a4..49daf5ae5b 100644 --- a/pypy/doc/releases/1.6.0.rst +++ b/pypy/doc/releases/1.6.0.rst @@ -30,7 +30,7 @@ layers which compose PyPy. In particular, we improved: the Garbage Collector, the JIT warmup time, the optimizations performed by the JIT, the quality of the generated machine code and the implementation of our Python interpreter. -.. _`pypy 1.5 and cpython 2.6.2`: http://speed.pypy.org +.. _pypy 1.5 and cpython 2.6.2: http://speed.pypy.org Highlights @@ -89,7 +89,7 @@ Hakan Ardo, Carl Friedrich Bolz, Laura Creighton, Antonio Cuni, Maciej Fijalkowski, Amaury Forgeot d'Arc, Alex Gaynor, Armin Rigo and the PyPy team -.. _`jitviewer`: http://morepypy.blogspot.com/2011/08/visualization-of-jitted-code.html -.. _`bug tracker`: https://bugs.pypy.org -.. _`compatibility wiki`: https://bitbucket.org/pypy/compatibility/wiki/Home +.. _jitviewer: http://morepypy.blogspot.com/2011/08/visualization-of-jitted-code.html +.. _bug tracker: https://bugs.pypy.org +.. _compatibility wiki: https://bitbucket.org/pypy/compatibility/wiki/Home diff --git a/pypy/doc/releases/1.7.0.rst b/pypy/doc/releases/1.7.0.rst index 6f8e447969..a6feb46d70 100644 --- a/pypy/doc/releases/1.7.0.rst +++ b/pypy/doc/releases/1.7.0.rst @@ -26,7 +26,7 @@ can greatly speed up. On average on our benchmark suite, PyPy 1.7 is around **30%** faster than PyPy 1.6 and up to **20 times** faster on some benchmarks. -.. _`pypy 1.7 and cpython 2.7.1`: http://speed.pypy.org +.. _pypy 1.7 and cpython 2.7.1: http://speed.pypy.org Highlights @@ -89,6 +89,6 @@ of those happen faster, we urge you to donate to `numpy proposal`_ or `py3k proposal`_. In case you want PyPy to progress, but you trust us with the general direction, you can always donate to the `general pot`_. -.. _`numpy proposal`: http://pypy.org/numpydonate.html -.. _`py3k proposal`: http://pypy.org/py3donate.html -.. _`general pot`: http://pypy.org +.. _numpy proposal: http://pypy.org/numpydonate.html +.. _py3k proposal: http://pypy.org/py3donate.html +.. _general pot: http://pypy.org diff --git a/pypy/doc/releases/1.8.0.rst b/pypy/doc/releases/1.8.0.rst index cad5251bbf..4dfb10e1eb 100644 --- a/pypy/doc/releases/1.8.0.rst +++ b/pypy/doc/releases/1.8.0.rst @@ -15,7 +15,7 @@ you can download the PyPy 1.8 release here: http://pypy.org/download.html -.. _`list strategies`: http://morepypy.blogspot.com/2011/10/more-compact-lists-with-list-strategies.html +.. _list strategies: http://morepypy.blogspot.com/2011/10/more-compact-lists-with-list-strategies.html What is PyPy? ============= @@ -28,7 +28,7 @@ This release supports x86 machines running Linux 32/64, Mac OS X 32/64 or Windows 32. Windows 64 work has been stalled, we would welcome a volunteer to handle that. -.. _`pypy 1.8 and cpython 2.7.1`: http://speed.pypy.org +.. _pypy 1.8 and cpython 2.7.1: http://speed.pypy.org Highlights @@ -90,9 +90,9 @@ the release or is not ready yet. Highlights include: Cheers, The PyPy Team -.. _`brief overview`: http://doc.pypy.org/en/latest/jit-hooks.html -.. _`numpy status page`: http://buildbot.pypy.org/numpy-status/latest.html -.. _`numpy status update blog report`: http://morepypy.blogspot.com/2012/01/numpypy-status-update.html -.. _`numpypy`: http://pypy.org/numpydonate.html -.. _`py3k`: http://pypy.org/py3donate.html -.. _`our plans`: http://morepypy.blogspot.com/2012/01/transactional-memory-ii.html +.. _brief overview: http://doc.pypy.org/en/latest/jit-hooks.html +.. _numpy status page: http://buildbot.pypy.org/numpy-status/latest.html +.. _numpy status update blog report: http://morepypy.blogspot.com/2012/01/numpypy-status-update.html +.. _numpypy: http://pypy.org/numpydonate.html +.. _py3k: http://pypy.org/py3donate.html +.. _our plans: http://morepypy.blogspot.com/2012/01/transactional-memory-ii.html diff --git a/pypy/doc/releases/1.9.0.rst b/pypy/doc/releases/1.9.0.rst index 5e41e74eae..6e848d08f1 100644 --- a/pypy/doc/releases/1.9.0.rst +++ b/pypy/doc/releases/1.9.0.rst @@ -9,9 +9,9 @@ It also brings an improved situation on Windows and OS X. You can download the PyPy 1.9 release here: - http://pypy.org/download.html + http://pypy.org/download.html -.. _`numpypy`: http://pypy.org/numpydonate.html +.. _numpypy: http://pypy.org/numpydonate.html What is PyPy? @@ -25,7 +25,7 @@ This release supports x86 machines running Linux 32/64, Mac OS X 64 or Windows 32. Windows 64 work is still stalling, we would welcome a volunteer to handle that. -.. _`pypy 1.9 and cpython 2.7.2`: http://speed.pypy.org +.. _pypy 1.9 and cpython 2.7.2: http://speed.pypy.org Thanks to our donors @@ -51,9 +51,9 @@ yet: people interested in them need to grab and (attempt to) translate PyPy from the corresponding branches (respectively ``py3k`` and ``stm-thread``). -.. _`NumPy in PyPy`: http://pypy.org/numpydonate.html -.. _`Py3k (Python 3)`: http://pypy.org/py3donate.html -.. _`Software Transactional Memory`: http://pypy.org/tmdonate.html +.. _NumPy in PyPy: http://pypy.org/numpydonate.html +.. _Py3k (Python 3): http://pypy.org/py3donate.html +.. _Software Transactional Memory: http://pypy.org/tmdonate.html Highlights ========== @@ -96,8 +96,8 @@ Highlights * List comprehension has been improved. -.. _`numpy-status`: http://buildbot.pypy.org/numpy-status/latest.html -.. _`JIT hooks documentation`: http://doc.pypy.org/en/latest/jit-hooks.html +.. _numpy-status: http://buildbot.pypy.org/numpy-status/latest.html +.. _JIT hooks documentation: http://doc.pypy.org/en/latest/jit-hooks.html JitViewer ========= @@ -105,7 +105,7 @@ JitViewer There will be a corresponding 1.9 release of JitViewer which is guaranteed to work with PyPy 1.9. See the `JitViewer docs`_ for details. -.. _`JitViewer docs`: http://bitbucket.org/pypy/jitviewer +.. _JitViewer docs: http://bitbucket.org/pypy/jitviewer Cheers, The PyPy Team diff --git a/pypy/doc/releases/2.0.0-beta1.rst b/pypy/doc/releases/2.0.0-beta1.rst index d5e091416d..f47ed2b5a1 100644 --- a/pypy/doc/releases/2.0.0-beta1.rst +++ b/pypy/doc/releases/2.0.0-beta1.rst @@ -14,7 +14,7 @@ numerous improvements to the numpy in pypy effort, cpyext and performance. You can download the PyPy 2.0 beta 1 release here: - http://pypy.org/download.html + http://pypy.org/download.html What is PyPy? ============= @@ -28,7 +28,7 @@ Windows 32. It also supports ARM machines running Linux. Windows 64 work is still stalling, we would welcome a volunteer to handle that. -.. _`pypy 2.0 beta 1 and cpython 2.7.3`: http://bit.ly/USXqpP +.. _pypy 2.0 beta 1 and cpython 2.7.3: http://bit.ly/USXqpP How to use PyPy? ================ @@ -37,10 +37,10 @@ We suggest using PyPy from a `virtualenv`_. Once you have a virtualenv installed, you can follow instructions from `pypy documentation`_ on how to proceed. This document also covers other `installation schemes`_. -.. _`pypy documentation`: http://doc.pypy.org/en/latest/getting-started.html#installing-using-virtualenv -.. _`virtualenv`: http://www.virtualenv.org/en/latest/ -.. _`installation schemes`: http://doc.pypy.org/en/latest/getting-started.html#installing-pypy -.. _`PyPy and pip`: http://doc.pypy.org/en/latest/getting-started.html#installing-pypy +.. _pypy documentation: http://doc.pypy.org/en/latest/getting-started.html#installing-using-virtualenv +.. _virtualenv: http://www.virtualenv.org/en/latest/ +.. _installation schemes: http://doc.pypy.org/en/latest/getting-started.html#installing-pypy +.. _PyPy and pip: http://doc.pypy.org/en/latest/getting-started.html#installing-pypy Regressions =========== @@ -101,8 +101,8 @@ Highlights unicode strings, which means that now such collections will be both faster and more compact. -.. _`cpython issue tracker`: http://bugs.python.org/issue14621 -.. _`jit hooks`: http://doc.pypy.org/en/latest/jit-hooks.html +.. _cpython issue tracker: http://bugs.python.org/issue14621 +.. _jit hooks: http://doc.pypy.org/en/latest/jit-hooks.html Things we're working on ======================= diff --git a/pypy/doc/releases/2.0.0-beta2.rst b/pypy/doc/releases/2.0.0-beta2.rst index 0e27de8523..eab90ac576 100644 --- a/pypy/doc/releases/2.0.0-beta2.rst +++ b/pypy/doc/releases/2.0.0-beta2.rst @@ -23,7 +23,7 @@ make it happen before the 2.0 final. The new major features are: You can download the PyPy 2.0 beta 2 release here: - http://pypy.org/download.html + http://pypy.org/download.html What is PyPy? ============= @@ -38,7 +38,7 @@ disabled for the beta 2 release. Windows 64 work is still stalling, we would welcome a volunteer to handle that. -.. _`pypy 2.0 beta 2 and cpython 2.7.3`: http://bit.ly/USXqpP +.. _pypy 2.0 beta 2 and cpython 2.7.3: http://bit.ly/USXqpP How to use PyPy? ================ @@ -47,9 +47,9 @@ We suggest using PyPy from a `virtualenv`_. Once you have a virtualenv installed, you can follow instructions from `pypy documentation`_ on how to proceed. This document also covers other `installation schemes`_. -.. _`pypy documentation`: http://doc.pypy.org/en/latest/getting-started.html#installing-using-virtualenv -.. _`virtualenv`: http://www.virtualenv.org/en/latest/ -.. _`installation schemes`: http://doc.pypy.org/en/latest/getting-started.html#installing-pypy +.. _pypy documentation: http://doc.pypy.org/en/latest/getting-started.html#installing-using-virtualenv +.. _virtualenv: http://www.virtualenv.org/en/latest/ +.. _installation schemes: http://doc.pypy.org/en/latest/getting-started.html#installing-pypy Highlights ========== @@ -83,12 +83,12 @@ Improvements since 1.9 unicode strings, which means that now such collections will be both faster and more compact. -.. _`eventlet`: http://eventlet.net/ -.. _`gevent`: http://www.gevent.org/ -.. _`cffi`: http://cffi.readthedocs.org/en/release-0.6/ -.. _`JIT hooks`: http://doc.pypy.org/en/latest/jit-hooks.html -.. _`pypycore`: https://github.com/gevent-on-pypy/pypycore -.. _`pypy-hacks`: https://github.com/schmir/gevent/tree/pypy-hacks -.. _`_curses.py`: https://bitbucket.org/pypy/pypy/src/aefddd47f224e3c12e2ea74f5c796d76f4355bdb/lib_pypy/_curses.py?at=default -.. _`_sqlite3.py`: https://bitbucket.org/pypy/pypy/src/aefddd47f224e3c12e2ea74f5c796d76f4355bdb/lib_pypy/_sqlite3.py?at=default +.. _eventlet: http://eventlet.net/ +.. _gevent: http://www.gevent.org/ +.. _cffi: http://cffi.readthedocs.org/en/release-0.6/ +.. _JIT hooks: http://doc.pypy.org/en/latest/jit-hooks.html +.. _pypycore: https://github.com/gevent-on-pypy/pypycore +.. _pypy-hacks: https://github.com/schmir/gevent/tree/pypy-hacks +.. _\_curses.py: https://bitbucket.org/pypy/pypy/src/aefddd47f224e3c12e2ea74f5c796d76f4355bdb/lib_pypy/_curses.py?at=default +.. _\_sqlite3.py: https://bitbucket.org/pypy/pypy/src/aefddd47f224e3c12e2ea74f5c796d76f4355bdb/lib_pypy/_sqlite3.py?at=default diff --git a/pypy/doc/releases/2.0.0.rst b/pypy/doc/releases/2.0.0.rst index 91ddd0f7b4..2fba55a019 100644 --- a/pypy/doc/releases/2.0.0.rst +++ b/pypy/doc/releases/2.0.0.rst @@ -17,7 +17,7 @@ The two biggest changes since PyPy 1.9 are: * PyPy now contains release 0.6 of `cffi`_ as a builtin module, which is preferred way of calling C from Python that works well on PyPy -.. _`cffi`: http://cffi.readthedocs.org +.. _cffi: http://cffi.readthedocs.org What is PyPy? ============= @@ -31,7 +31,7 @@ Windows 32. Windows 64 work is still stalling, we would welcome a volunteer to handle that. ARM support is on the way and we're expecting to release an alpha ARM version shortly. -.. _`pypy 2.0 and cpython 2.7.3`: http://speed.pypy.org +.. _pypy 2.0 and cpython 2.7.3: http://speed.pypy.org Highlights ========== @@ -54,8 +54,8 @@ Highlights * A lot of stability issues fixed. -.. _`pypycore`: https://github.com/gevent-on-pypy/pypycore/ -.. _`pypy-hacks`: https://github.com/schmir/gevent/tree/pypy-hacks +.. _pypycore: https://github.com/gevent-on-pypy/pypycore/ +.. _pypy-hacks: https://github.com/schmir/gevent/tree/pypy-hacks Cheers, fijal, arigo and the PyPy team diff --git a/pypy/doc/sandbox.rst b/pypy/doc/sandbox.rst index 86eaafed25..a3ac1955c5 100644 --- a/pypy/doc/sandbox.rst +++ b/pypy/doc/sandbox.rst @@ -26,11 +26,11 @@ this case we also generate systematic run-time checks against buffer overflows. .. warning:: - - The hard work from the PyPy side is done --- you get a fully secure - version. What is only experimental and unpolished is the library to - use this sandboxed PyPy from a regular Python interpreter (CPython, or - an unsandboxed PyPy). Contributions welcome. + + The hard work from the PyPy side is done --- you get a fully secure + version. What is only experimental and unpolished is the library to + use this sandboxed PyPy from a regular Python interpreter (CPython, or + an unsandboxed PyPy). Contributions welcome. Overview @@ -39,7 +39,7 @@ Overview One of PyPy's translation aspects is a sandboxing feature. It's "sandboxing" as in "full virtualization", but done in normal C with no OS support at all. It's a two-processes model: we can translate PyPy to a special "pypy-c-sandbox" -executable, which is safe in the sense that it doesn't do any library or +executable, which is safe in the sense that it doesn't do any library or system calls - instead, whenever it would like to perform such an operation, it marshals the operation name and the arguments to its stdout and it waits for the marshalled result on its stdin. This pypy-c-sandbox process is meant to be @@ -83,10 +83,10 @@ virtual directories like /bin/lib/pypy1.2/lib-python and /bin/lib/pypy1.2/lib_pypy and it pretends that the executable is /bin/pypy-c. + Howto ----- - In pypy/goal:: ../../rpython/bin/rpython -O2 --sandbox targetpypystandalone.py diff --git a/pypy/doc/sprint-reports.rst b/pypy/doc/sprint-reports.rst index d1980795a9..d3ed18bf71 100644 --- a/pypy/doc/sprint-reports.rst +++ b/pypy/doc/sprint-reports.rst @@ -1,7 +1,7 @@ Sprint reports from PyPy sprints 2003-2006 ========================================== -Here are links to sprint reports from various sprints in the PyPy project, +Here are links to sprint reports from various sprints in the PyPy project, an informal summary of features being worked on and the results of that work. A good summary of the progress over the years.....enjoy! @@ -13,7 +13,7 @@ A good summary of the progress over the years.....enjoy! * Europython/Gothenburg (June 2004) * `Vilnius (Nov 2004)`_ * `Leysin (Jan 2005)`_ - * `PyCon/Washington (March 2005)`_ + * `PyCon/Washington (March 2005)`_ * `Europython/Gothenburg (June 2005)`_ * `Hildesheim (July 2005)`_ * `Heidelberg (Aug 2005)`_ @@ -42,40 +42,38 @@ A good summary of the progress over the years.....enjoy! * `CERN (July 2010)`_ * `Düsseldorf (October 2010)`_ - .. _Hildesheim (Feb 2003): http://codespeak.net/pypy/extradoc/sprintinfo/HildesheimReport.html - .. _Gothenburg (May 2003): http://codespeak.net/pypy/extradoc/sprintinfo/gothenburg-2003-sprintreport.txt - .. _LovainLaNeuve (June 2003): http://codespeak.net/pypy/extradoc/sprintinfo/LouvainLaNeuveReport.txt - .. _Berlin (Sept 2003): http://codespeak.net/pypy/extradoc/sprintinfo/BerlinReport.txt - .. _Amsterdam (Dec 2003): http://codespeak.net/pypy/extradoc/sprintinfo/AmsterdamReport.txt - .. _Vilnius (Nov 2004): http://codespeak.net/pypy/extradoc/sprintinfo/vilnius-2004-sprintreport.txt - .. _Leysin (Jan 2005): http://codespeak.net/pypy/extradoc/sprintinfo/LeysinReport.txt - .. _PyCon/Washington (March 2005): http://codespeak.net/pypy/extradoc/sprintinfo/pycon_sprint_report.txt - .. _Europython/Gothenburg (June 2005): http://codespeak.net/pypy/extradoc/sprintinfo/ep2005-sprintreport.txt - .. _Hildesheim (July 2005): http://codespeak.net/pypy/extradoc/sprintinfo/hildesheim2005-sprintreport.txt - .. _Heidelberg (Aug 2005): http://codespeak.net/pypy/extradoc/sprintinfo/Heidelberg-report.txt - .. _Paris (Oct 2005): http://codespeak.net/pypy/extradoc/sprintinfo/paris/paris-report.txt - .. _Gothenburg (Dec 2005): http://codespeak.net/pypy/extradoc/sprintinfo/gothenburg-2005/gothenburg-dec2005-sprintreport.txt - .. _Mallorca (Jan 2006): http://codespeak.net/pypy/extradoc/sprintinfo/mallorca/mallorca-sprintreport.txt - .. _LouvainLaNeuve (March 2006): http://codespeak.net/pypy/extradoc/sprintinfo/louvain-la-neuve-2006/report.txt - .. _Leysin (April 2006): http://codespeak.net/pypy/extradoc/sprintinfo/leysin-winter-2006-sprintreport.txt - .. _Tokyo (April 2006): http://codespeak.net/pypy/extradoc/sprintinfo/tokyo/sprint-report.txt - .. _Düsseldorf (June 2006): http://codespeak.net/pypy/extradoc/sprintinfo/ddorf2006/report1.txt - .. _Europython/Geneva (July 2006): http://codespeak.net/pypy/extradoc/sprintinfo/post-ep2006/report.txt - .. _Düsseldorf (October 2006): http://codespeak.net/pypy/extradoc/sprintinfo/ddorf2006b/report.txt - .. _`Leysin (January 2007)`: http://codespeak.net/pypy/extradoc/sprintinfo/leysin-winter-2007/report.txt - .. _Hildesheim (Feb 2007): http://codespeak.net/pypy/extradoc/sprintinfo/trillke-2007/sprint-report.txt - .. _`EU report writing sprint`: http://codespeak.net/pypy/extradoc/sprintinfo/trillke-2007/eu-report-sprint-report.txt - .. _`PyCon/Dallas (Feb 2006)`: http://codespeak.net/pypy/extradoc/sprintinfo/pycon06/sprint-report.txt - .. _`Göteborg (November 2007)`: http://morepypy.blogspot.com/2007_11_01_archive.html - .. _`Leysin (January 2008)`: http://morepypy.blogspot.com/2008/01/leysin-winter-sport-sprint-started.html - .. _`Berlin (May 2008)`: http://morepypy.blogspot.com/2008_05_01_archive.html - .. _`Vilnius after EuroPython (July 2008)`: http://morepypy.blogspot.com/2008/07/europython-2008-pypy-talks-and-sprint.html - .. _`Düsseldorf (August 2008)`: http://morepypy.blogspot.com/2008_10_01_archive.html - .. _`Wroclaw (February 2009)`: http://morepypy.blogspot.com/2009/02/wroclaw-2009-sprint-progress-report.html - .. _`Leysin (April 2009)`: http://morepypy.blogspot.com/2009/04/leysin-sprint-report.html - .. _`Göteborg (August 2009)`: http://morepypy.blogspot.com/2009/08/gothenburg-jit-sprint-report.html - .. _`Düsseldorf (November 2009)`: http://morepypy.blogspot.com/2009/11/dusseldorf-sprint-report.html - .. _`CERN (July 2010)`: http://morepypy.blogspot.com/2010/07/cern-sprint-report-wrapping-c-libraries.html - .. _`Düsseldorf (October 2010)`: http://morepypy.blogspot.com/2010/10/dusseldorf-sprint-report-2010.html - - +.. _Hildesheim (Feb 2003): http://codespeak.net/pypy/extradoc/sprintinfo/HildesheimReport.html +.. _Gothenburg (May 2003): http://codespeak.net/pypy/extradoc/sprintinfo/gothenburg-2003-sprintreport.txt +.. _LovainLaNeuve (June 2003): http://codespeak.net/pypy/extradoc/sprintinfo/LouvainLaNeuveReport.txt +.. _Berlin (Sept 2003): http://codespeak.net/pypy/extradoc/sprintinfo/BerlinReport.txt +.. _Amsterdam (Dec 2003): http://codespeak.net/pypy/extradoc/sprintinfo/AmsterdamReport.txt +.. _Vilnius (Nov 2004): http://codespeak.net/pypy/extradoc/sprintinfo/vilnius-2004-sprintreport.txt +.. _Leysin (Jan 2005): http://codespeak.net/pypy/extradoc/sprintinfo/LeysinReport.txt +.. _PyCon/Washington (March 2005): http://codespeak.net/pypy/extradoc/sprintinfo/pycon_sprint_report.txt +.. _Europython/Gothenburg (June 2005): http://codespeak.net/pypy/extradoc/sprintinfo/ep2005-sprintreport.txt +.. _Hildesheim (July 2005): http://codespeak.net/pypy/extradoc/sprintinfo/hildesheim2005-sprintreport.txt +.. _Heidelberg (Aug 2005): http://codespeak.net/pypy/extradoc/sprintinfo/Heidelberg-report.txt +.. _Paris (Oct 2005): http://codespeak.net/pypy/extradoc/sprintinfo/paris/paris-report.txt +.. _Gothenburg (Dec 2005): http://codespeak.net/pypy/extradoc/sprintinfo/gothenburg-2005/gothenburg-dec2005-sprintreport.txt +.. _Mallorca (Jan 2006): http://codespeak.net/pypy/extradoc/sprintinfo/mallorca/mallorca-sprintreport.txt +.. _LouvainLaNeuve (March 2006): http://codespeak.net/pypy/extradoc/sprintinfo/louvain-la-neuve-2006/report.txt +.. _Leysin (April 2006): http://codespeak.net/pypy/extradoc/sprintinfo/leysin-winter-2006-sprintreport.txt +.. _Tokyo (April 2006): http://codespeak.net/pypy/extradoc/sprintinfo/tokyo/sprint-report.txt +.. _Düsseldorf (June 2006): http://codespeak.net/pypy/extradoc/sprintinfo/ddorf2006/report1.txt +.. _Europython/Geneva (July 2006): http://codespeak.net/pypy/extradoc/sprintinfo/post-ep2006/report.txt +.. _Düsseldorf (October 2006): http://codespeak.net/pypy/extradoc/sprintinfo/ddorf2006b/report.txt +.. _Leysin (January 2007): http://codespeak.net/pypy/extradoc/sprintinfo/leysin-winter-2007/report.txt +.. _Hildesheim (Feb 2007): http://codespeak.net/pypy/extradoc/sprintinfo/trillke-2007/sprint-report.txt +.. _EU report writing sprint: http://codespeak.net/pypy/extradoc/sprintinfo/trillke-2007/eu-report-sprint-report.txt +.. _PyCon/Dallas (Feb 2006): http://codespeak.net/pypy/extradoc/sprintinfo/pycon06/sprint-report.txt +.. _Göteborg (November 2007): http://morepypy.blogspot.com/2007_11_01_archive.html +.. _Leysin (January 2008): http://morepypy.blogspot.com/2008/01/leysin-winter-sport-sprint-started.html +.. _Berlin (May 2008): http://morepypy.blogspot.com/2008_05_01_archive.html +.. _Vilnius after EuroPython (July 2008): http://morepypy.blogspot.com/2008/07/europython-2008-pypy-talks-and-sprint.html +.. _Düsseldorf (August 2008): http://morepypy.blogspot.com/2008_10_01_archive.html +.. _Wroclaw (February 2009): http://morepypy.blogspot.com/2009/02/wroclaw-2009-sprint-progress-report.html +.. _Leysin (April 2009): http://morepypy.blogspot.com/2009/04/leysin-sprint-report.html +.. _Göteborg (August 2009): http://morepypy.blogspot.com/2009/08/gothenburg-jit-sprint-report.html +.. _Düsseldorf (November 2009): http://morepypy.blogspot.com/2009/11/dusseldorf-sprint-report.html +.. _CERN (July 2010): http://morepypy.blogspot.com/2010/07/cern-sprint-report-wrapping-c-libraries.html +.. _Düsseldorf (October 2010): http://morepypy.blogspot.com/2010/10/dusseldorf-sprint-report-2010.html diff --git a/pypy/doc/stackless.rst b/pypy/doc/stackless.rst index f2cee35e0d..4adb72ed8e 100644 --- a/pypy/doc/stackless.rst +++ b/pypy/doc/stackless.rst @@ -1,11 +1,8 @@ -========================================================== - Application-level Stackless features -========================================================== - - +Application-level Stackless features +==================================== Introduction -================ +------------ PyPy can expose to its user language features similar to the ones present in `Stackless Python`_: the ability to write code in a @@ -31,9 +28,11 @@ only available (so far) on x86 and x86-64 CPUs; for other CPUs you need to add a short page of custom assembler to :source:`rpython/translator/c/src/stacklet/`. +.. _Stackless Python: http://www.stackless.com + Theory -====== +------ The fundamental idea is that, at any point in time, the program happens to run one stack of frames (or one per thread, in case of @@ -75,13 +74,12 @@ tracebacks too). Application level interface -============================= - +--------------------------- .. _continulet: Continulets -+++++++++++ +~~~~~~~~~~~ A translated PyPy contains by default a module called ``_continuation`` exporting the type ``continulet``. A ``continulet`` object from this @@ -156,7 +154,7 @@ In more details: Genlets -+++++++ +~~~~~~~ The ``_continuation`` module also exposes the ``generator`` decorator:: @@ -183,7 +181,7 @@ The ``generator`` decorator can also be applied to methods:: Greenlets -+++++++++ +~~~~~~~~~ Greenlets are implemented on top of continulets in :source:`lib_pypy/greenlet.py`. See the official `documentation of the greenlets`_. @@ -192,9 +190,11 @@ Note that unlike the CPython greenlets, this version does not suffer from GC issues: if the program "forgets" an unfinished greenlet, it will always be collected at the next garbage collection. +.. _documentation of the greenlets: http://packages.python.org/greenlet/ + Unimplemented features -++++++++++++++++++++++ +~~~~~~~~~~~~~~~~~~~~~~ The following features (present in some past Stackless version of PyPy) are for the time being not supported any more: @@ -213,7 +213,7 @@ are for the time being not supported any more: Recursion depth limit -+++++++++++++++++++++ +~~~~~~~~~~~~~~~~~~~~~ You can use continulets to emulate the infinite recursion depth present in Stackless Python and in stackless-enabled older versions of PyPy. @@ -266,7 +266,7 @@ to use other interfaces like genlets and greenlets.) Stacklets -+++++++++ +~~~~~~~~~ Continulets are internally implemented using stacklets, which is the generic RPython-level building block for "one-shot continuations". For @@ -284,7 +284,7 @@ wrapper like PyPy's '_continuation' module. Theory of composability -+++++++++++++++++++++++ +~~~~~~~~~~~~~~~~~~~~~~~ Although the concept of coroutines is far from new, they have not been generally integrated into mainstream languages, or only in limited form @@ -407,7 +407,3 @@ that switched to it) belong to the same class or module; so from that point of view the continulet is a purely local link between two local frames. It doesn't make sense to have a concept that allows this link to be manipulated from outside. - - -.. _`Stackless Python`: http://www.stackless.com -.. _`documentation of the greenlets`: http://packages.python.org/greenlet/ diff --git a/pypy/doc/statistic/index.rst b/pypy/doc/statistic/index.rst index 62dc9f7c61..10b4a6062c 100644 --- a/pypy/doc/statistic/index.rst +++ b/pypy/doc/statistic/index.rst @@ -63,5 +63,5 @@ Page hits to http://codespeak.net/pypy/: .. image:: webaccess.png -.. _`pypy-dev`: http://python.org/mailman/listinfo/pypy-commit -.. _`pypy-svn`: http://python.org/mailman/listinfo/pypy-dev +.. _pypy-dev: http://python.org/mailman/listinfo/pypy-commit +.. _pypy-svn: http://python.org/mailman/listinfo/pypy-dev diff --git a/pypy/doc/tool/makeref.py b/pypy/doc/tool/makeref.py deleted file mode 100644 index ca13f836f3..0000000000 --- a/pypy/doc/tool/makeref.py +++ /dev/null @@ -1,64 +0,0 @@ - -import py -import pypy -pypydir = py.path.local(pypy.__file__).join('..') -distdir = pypydir.dirpath() -issue_url = 'http://bugs.pypy.org/issue/pypy-dev/' -bitbucket_url = 'https://bitbucket.org/pypy/pypy/src/default/' - -import urllib2, posixpath - - -def makeref(docdir): - reffile = docdir.join('_ref.txt') - - linkrex = py.std.re.compile('`(\S+)`_') - - name2target = {} - def addlink(linkname, linktarget): - assert linkname and linkname != '/' - if linktarget in name2target: - if linkname in name2target[linktarget]: - return - name2target.setdefault(linktarget, []).append(linkname) - - for textfile in sorted(docdir.listdir()): # for subdirs, see below - if textfile.ext != '.rst': - continue - content = textfile.read() - found = False - for linkname in linkrex.findall(content): - if '/' in linkname: - found = True - assert distdir.join(linkname).check(), "link %s in %s is dead" % (linkname, textfile) - url = bitbucket_url + linkname - if not linkname.endswith("/") and distdir.join(linkname).check(dir=1): - url += "/" - addlink(linkname, url) - elif linkname.startswith('issue'): - found = True - addlink(linkname, issue_url+linkname) - if found: - assert ".. include:: _ref.txt" in content, "you need to include _ref.txt in %s" % (textfile, ) - - items = name2target.items() - items.sort() - - lines = [] - for linktarget, linknamelist in items: - linknamelist.sort() - for linkname in linknamelist[:-1]: - lines.append(".. _`%s`:" % linkname) - lines.append(".. _`%s`: %s" %(linknamelist[-1], linktarget)) - - lines.append('') - reffile.write("\n".join(lines)) - print "wrote %d references to %r" %(len(lines), reffile) - #print "last ten lines" - #for x in lines[-10:]: print x - - -# We need to build a new _ref.txt for each directory that uses it, because -# they differ in the number of "../" that they need in the link targets... -makeref(pypydir.join('doc')) -makeref(pypydir.join('doc').join('jit')) diff --git a/pypy/doc/video-index.rst b/pypy/doc/video-index.rst index ba56f76a98..d0ec691b33 100644 --- a/pypy/doc/video-index.rst +++ b/pypy/doc/video-index.rst @@ -1,14 +1,13 @@ -========================= -PyPy video documentation -========================= +PyPy video documentation +======================== -Copyrights and Licensing ----------------------------- +Copyrights and Licensing +------------------------ The following videos are copyrighted by merlinux gmbh and available on YouTube. -If you need another license, don't hesitate to contact us. +If you need another license, don't hesitate to contact us. Trailer: PyPy at the PyCon 2006 @@ -212,4 +211,3 @@ translation toolchain and the just-in-time compiler. <iframe width="420" height="315" src="http://www.youtube.com/embed/6dnUzVQaSlg?rel=0" frameborder="0" allowfullscreen></iframe> - diff --git a/pypy/doc/you-want-to-help.rst b/pypy/doc/you-want-to-help.rst index 0f95be8e7d..2273b5d63f 100644 --- a/pypy/doc/you-want-to-help.rst +++ b/pypy/doc/you-want-to-help.rst @@ -1,4 +1,3 @@ - You want to help with PyPy, now what? ===================================== @@ -24,12 +23,13 @@ important lessons that everyone willing to contribute should learn: * PyPy uses an entirely different set of tools - most of them included in the PyPy repository. There is no Makefile, nor autoconf. More below + Architecture -============ +------------ PyPy has layers. The 100 miles view: -* `RPython`_ is the language in which we write interpreters. Not the entire +* :doc:`RPython <rpython:rpython>` is the language in which we write interpreters. Not the entire PyPy project is written in RPython, only the parts that are compiled in the translation process. The interesting point is that RPython has no parser, it's compiled from the live python objects, which make it possible to do @@ -38,16 +38,12 @@ PyPy has layers. The 100 miles view: The RPython standard library is to be found in the ``rlib`` subdirectory. -.. _`RPython`: coding-guide.html#RPython - * The translation toolchain - this is the part that takes care about translating - RPython to flow graphs and then to C. There is more in the `architecture`_ + RPython to flow graphs and then to C. There is more in the :doc:`architecture <architecture>` document written about it. It mostly lives in ``rpython``, ``annotator`` and ``objspace/flow``. -.. _`architecture`: architecture.html - * Python Interpreter xxx @@ -56,7 +52,7 @@ PyPy has layers. The 100 miles view: xxx -* Just-in-Time Compiler (JIT): `we have a tracing JIT`_ that traces the +* :doc:`Just-in-Time Compiler (JIT) <rpython:jit/index>`: we have a tracing JIT that traces the interpreter written in RPython, rather than the user program that it interprets. As a result it applies to any interpreter, i.e. any language. But getting it to work correctly is not trivial: it @@ -68,19 +64,15 @@ PyPy has layers. The 100 miles view: that turns it into machine code. Writing a new backend is a traditional way to get into the project. -.. _`we have a tracing JIT`: jit/index.html - * Garbage Collectors (GC): as you can notice if you are used to CPython's C code, there are no ``Py_INCREF/Py_DECREF`` equivalents in RPython code. - `Garbage collection in PyPy`_ is inserted + :doc:`rpython:garbage-collection` is inserted during translation. Moreover, this is not reference counting; it is a real GC written as more RPython code. The best one we have so far is in ``rpython/memory/gc/minimark.py``. -.. _`Garbage collection in PyPy`: garbage_collection.html - Toolset -======= +------- xxx diff --git a/pypy/tool/rest/rst.py b/pypy/tool/rest/rst.py index f5affe42d8..08514fe2a9 100644 --- a/pypy/tool/rest/rst.py +++ b/pypy/tool/rest/rst.py @@ -121,7 +121,7 @@ class Rest(AbstractNode): link_texts = [] # XXX this could check for duplicates and remove them... for link, target in self.links.iteritems(): - link_texts.append(".. _`%s`: %s" % (escape(link), target)) + link_texts.append(".. _%s: %s" % (escape(link), target)) return "\n" + "\n".join(link_texts) + "\n\n" def text(self): @@ -379,7 +379,7 @@ class LinkTarget(Paragraph): self.target = target def text(self): - return ".. _`%s`:%s\n" % (self.name, self.target) + return ".. _%s: %s\n" % (self.name, self.target) class Substitution(AbstractText): def __init__(self, text, **kwargs): diff --git a/rpython/doc/architecture.rst b/rpython/doc/architecture.rst index c9f2b3d135..d4da68cb0e 100644 --- a/rpython/doc/architecture.rst +++ b/rpython/doc/architecture.rst @@ -116,7 +116,7 @@ This process is described in much more detail in the :doc:`document about the RPython toolchain <translation>` and in the paper `Compiling dynamic language implementations`_. -.. _`Compiling dynamic language implementations`: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D05.1_Publish_on_translating_a_very-high-level_description.pdf +.. _Compiling dynamic language implementations: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D05.1_Publish_on_translating_a_very-high-level_description.pdf Further reading --------------- @@ -137,5 +137,5 @@ Further reading * A tutorial of how to use the :doc:`RPython toolchain <translation>` to `implement your own interpreter`_. -.. _`PyPy's approach to virtual machine construction`: https://bitbucket.org/pypy/extradoc/raw/tip/talk/dls2006/pypy-vm-construction.pdf -.. _`implement your own interpreter`: http://morepypy.blogspot.com/2011/04/tutorial-writing-interpreter-with-pypy.html +.. _PyPy's approach to virtual machine construction: https://bitbucket.org/pypy/extradoc/raw/tip/talk/dls2006/pypy-vm-construction.pdf +.. _implement your own interpreter: http://morepypy.blogspot.com/2011/04/tutorial-writing-interpreter-with-pypy.html diff --git a/rpython/doc/arm.rst b/rpython/doc/arm.rst index 7daac42566..5d73d6a863 100644 --- a/rpython/doc/arm.rst +++ b/rpython/doc/arm.rst @@ -1,8 +1,6 @@ -========================= Cross-translating for ARM ========================= - Here we describe the setup required and the steps needed to follow to translate an interpreter using the RPython translator to target ARM using a cross compilation toolchain. @@ -29,7 +27,7 @@ The tools required to cross translate from a Linux based host to an ARM based Li - A 32-bit PyPy or Python. - And the following (or corresponding) packages need to be installed to create an ARM based chroot: - * ``debootstrap`` + * ``debootstrap`` * ``schroot`` * ``binfmt-support`` * ``qemu-system`` @@ -52,7 +50,7 @@ Create the rootfs by calling: Next, copy the qemu-arm-static binary to the rootfs. -:: +:: cp /usr/bin/qemu-arm-static /srv/chroot/precise_arm/usr/bin/qemu-arm-static @@ -74,13 +72,12 @@ schroot configuration file in /etc/schroot/schroot.conf. To verify that everything is working in the chroot, running ``schroot -c precise_arm`` should start a shell running in the schroot environment using -qemu-arm to execute the ARM binaries. Running ``uname -m`` in the chroot should -yeild a result like ``armv7l``. Showing that we are emulating an ARM system. +qemu-arm to execute the ARM binaries. Running ``uname -m`` in the chroot should +yield a result like ``armv7l``. Showing that we are emulating an ARM system. Start the schroot as the user root in order to configure the apt sources and to install the following packages: - :: schroot -c precise_arm -u root @@ -124,13 +121,12 @@ contain the command line options for the sb2 command. If our rootfs is in the folder /srv/chroot/precise_arm and the scratchbox environment is called "ARM", the variables would be defined as follows. - :: export SB2=/srv/chroot/precise_arm export SB2OPT='-t ARM' -Once this is set, you can call the translator. For example save this file +Once this is set, you can call the translator. For example save this file :: @@ -149,7 +145,7 @@ and call the translator If everything worked correctly this should yield an ARM binary. Running this binary in the ARM chroot or on an ARM device should produce the output ``"Hello World"``. -To translate the full python pypy interpreter with a jit, you can cd into pypy/goal and call +To translate the full python pypy interpreter with a jit, you can cd into pypy/goal and call :: @@ -157,5 +153,5 @@ To translate the full python pypy interpreter with a jit, you can cd into pypy/g The gcrootfinder option is needed to work around `issue 1377`_ and the jit-backend works around `issue 1376`_ -.. _`issue 1377`: https://bugs.pypy.org/issue1377 -.. _`issue 1376`: https://bugs.pypy.org/issue1376 +.. _issue 1377: https://bugs.pypy.org/issue1377 +.. _issue 1376: https://bugs.pypy.org/issue1376 diff --git a/rpython/doc/cli-backend.rst b/rpython/doc/cli-backend.rst index 8bd43aec31..0a2174ce88 100644 --- a/rpython/doc/cli-backend.rst +++ b/rpython/doc/cli-backend.rst @@ -1,4 +1,3 @@ -=============== The CLI backend =============== @@ -7,7 +6,7 @@ machine. Target environment and language -=============================== +------------------------------- The target of GenCLI is the Common Language Infrastructure environment as defined by the `Standard Ecma 335`_. @@ -38,7 +37,7 @@ high level approach has two main advantages: optimizations. In reality the first point is not an advantage in the PyPy context, -because the `flow graph`_ we start from is quite low level and Python +because the :ref:`flow graph <flow-model>` we start from is quite low level and Python loops are already expressed in terms of branches (i.e., gotos). About the compiler optimizations we must remember that the flow graph @@ -63,9 +62,11 @@ generate: - directly generate code on the fly by accessing the facilities exposed by the System.Reflection.Emit API. +.. _Standard Ecma 335: http://www.ecma-international.org/publications/standards/Ecma-335.htm + Handling platform differences -============================= +----------------------------- Since our goal is to support both Microsoft CLR we have to handle the differences between the twos; in particular the main differences are @@ -92,7 +93,7 @@ everything except the assembling phase, for which we use Mono's Targeting the CLI Virtual Machine -================================= +--------------------------------- In order to write a CLI backend we have to take a number of decisions. First, we have to choose the typesystem to use: given that CLI @@ -117,9 +118,9 @@ to do for completing the backend: Mapping primitive types ------------------------ +~~~~~~~~~~~~~~~~~~~~~~~ -The `rtyper`_ give us a flow graph annotated with types belonging to +:doc:`rtyper` give us a flow graph annotated with types belonging to ootypesystem: in order to produce CLI code we need to translate these types into their Common Type System equivalents. @@ -146,8 +147,9 @@ least two ways to map plain Char to CTS: We think that mapping Python strings to .NET strings is fundamental, so we chose the second option. + Mapping built-in types ----------------------- +~~~~~~~~~~~~~~~~~~~~~~ As we saw in section ootypesystem defines a set of types that take advantage of built-in types offered by the platform. @@ -180,7 +182,7 @@ by explicitly passing the string object in the argument list. Mapping instructions --------------------- +~~~~~~~~~~~~~~~~~~~~ PyPy's low level operations are expressed in Static Single Information (SSI) form, such as this:: @@ -220,8 +222,9 @@ C#. The code that implements the mapping is in the modules opcodes.py. + Mapping exceptions ------------------- +~~~~~~~~~~~~~~~~~~ Both RPython and CLI have their own set of exception classes: some of these are pretty similar; e.g., we have OverflowError, @@ -261,7 +264,7 @@ overflows, catch that exception and throw a new one:: Translating flow graphs ------------------------ +~~~~~~~~~~~~~~~~~~~~~~~ As we saw previously in PyPy function and method bodies are represented by flow graphs that we need to translate CLI IL code. Flow @@ -337,7 +340,7 @@ exception handling machinery. Translating classes -------------------- +~~~~~~~~~~~~~~~~~~~ As we saw previously, the semantic of ootypesystem classes is very similar to the .NET one, so the translation is mostly @@ -363,8 +366,9 @@ Inheritance is straightforward too, as it is natively supported by CLI. The only noticeable thing is that we map ootypesystem's ROOT class to the CLI equivalent System.Object. + The Runtime Environment ------------------------ +~~~~~~~~~~~~~~~~~~~~~~~ The runtime environment is a collection of helper classes and functions used and referenced by many of the GenCLI submodules. It is @@ -386,7 +390,7 @@ the third is in the pypy.test one. Testing GenCLI -============== +-------------- As the rest of PyPy, GenCLI is a test-driven project: there is at least one unit test for almost each single feature of the @@ -425,8 +429,9 @@ r_longlong, r_ulonglong, bool, float and one-length strings (i.e., chars). By contrast, most types are fine for being returned: these include all primitive types, list, tuples and instances. + Installing Python for .NET on Linux -=================================== +----------------------------------- With the CLI backend, you can access .NET libraries from RPython; programs using .NET libraries will always run when translated, but you @@ -448,8 +453,4 @@ following lines inside the file (assuming you are using Python 2.7):: The installation should be complete now. To run Python for .NET, simply type ``mono python.exe``. - -.. _`Standard Ecma 335`: http://www.ecma-international.org/publications/standards/Ecma-335.htm -.. _`flow graph`: translation.html#the-flow-model -.. _`rtyper`: rtyper.html -.. _`Python for .NET`: http://pythonnet.sourceforge.net/ +.. _Python for .NET: http://pythonnet.sourceforge.net/ diff --git a/rpython/doc/conf.py b/rpython/doc/conf.py index 7b8b6eeb76..9e66d29830 100644 --- a/rpython/doc/conf.py +++ b/rpython/doc/conf.py @@ -25,7 +25,7 @@ import sys, os # Add any Sphinx extension module names here, as strings. They can be extensions # coming with Sphinx (named 'sphinx.ext.*') or your custom ones. -extensions = ['sphinx.ext.extlinks', 'sphinx.ext.graphviz'] +extensions = ['sphinx.ext.extlinks', 'sphinx.ext.intersphinx', 'sphinx.ext.graphviz'] # Add any paths that contain templates here, relative to this directory. templates_path = ['_templates'] @@ -208,6 +208,10 @@ latex_documents = [ #latex_domain_indices = True +intersphinx_mapping = {'python': ('http://docs.python.org/2.7', None), + 'pypy': ('http://pypy.readthedocs.org/en/improve-docs/', None)} + + # -- Options for manual page output -------------------------------------------- # One entry per manual page. List of tuples diff --git a/rpython/doc/faq.rst b/rpython/doc/faq.rst index 7ebe154197..13f9ada9da 100644 --- a/rpython/doc/faq.rst +++ b/rpython/doc/faq.rst @@ -1,17 +1,15 @@ -========================== Frequently Asked Questions ========================== .. contents:: ----------------- What is RPython? ---------------- RPython is a framework for implementing interpreters and virtual machines for programming languages, especially dynamic languages. ------------------------------------------------- + Can RPython compile normal Python programs to C? ------------------------------------------------ @@ -29,9 +27,9 @@ If you want a fast Python program, please use the PyPy JIT_ instead. Localized Type Inference of Atomic Types in Python, http://citeseer.ist.psu.edu/viewdoc/summary?doi=10.1.1.90.3231 -.. _`PyPy's RPython`: ------------------------------- +.. _PyPy's RPython: + What is this RPython language? ------------------------------ @@ -58,7 +56,7 @@ some functions in ``os``, ``math`` and ``time`` that have native support. To read more about the RPython limitations read the :doc:`RPython description <rpython>`. ---------------------------------------------------------------- + Does RPython have anything to do with Zope's Restricted Python? --------------------------------------------------------------- @@ -67,10 +65,10 @@ execution environment for CPython. `PyPy's RPython`_ is the implementation language for dynamic language interpreters. However, PyPy also provides a robust `sandboxed Python Interpreter`_. -.. _`sandboxed Python Interpreter`: sandbox.html -.. _`Zope's RestrictedPython`: http://pypi.python.org/pypi/RestrictedPython +.. _sandboxed Python Interpreter: sandbox.html +.. _Zope's RestrictedPython: http://pypi.python.org/pypi/RestrictedPython + ------------------------------------------------------- What's the ``"NOT_RPYTHON"`` I see in some docstrings? ------------------------------------------------------ @@ -80,7 +78,6 @@ stops and reports this as an error. You can therefore mark functions as "NOT_RPYTHON" to make sure that they are never analyzed. -------------------------------------------------------------------- Couldn't we simply take a Python syntax tree and turn it into Lisp? ------------------------------------------------------------------- @@ -100,7 +97,7 @@ You could make ``#'py:add`` a generic function and see if a given CLOS implementation is fast enough to give a useful speed (but I think the coercion rules would probably drive you insane first). -- mwh --------------------------------------------- + Do I have to rewrite my programs in RPython? -------------------------------------------- @@ -111,7 +108,7 @@ things", like use *any* part of the standard Python library or *any* 3rd-party library, then it is not RPython to start with. You should only look at RPython if you try to `write your own interpreter`__. -.. __: `how do I compile my own interpreters`_ +.. __: `How do I compile my own interpreters?`_ If your goal is to speed up Python code, then look at the regular PyPy, which is a full and compliant Python 2.7 interpreter (which happens to @@ -125,7 +122,7 @@ this case is not interesting for us. If you needed to rewrite the code in RPython, you could as well have rewritten it in C for example. The latter is a much more supported, much more documented language `:-)` ---------------------------------------------------- + Which backends are there for the RPython toolchain? --------------------------------------------------- @@ -136,9 +133,9 @@ To learn more about backends take a look at the `translation document`_. .. _C: translation.html#the-c-back-end .. _CLI: cli-backend.html .. _JVM: translation.html#genjvm -.. _`translation document`: translation.html +.. _translation document: translation.html + ------------------- Could we use LLVM? ------------------ @@ -162,24 +159,17 @@ make an N+1'th attempt with LLVM, he is welcome, and he will receive a bit of help on the IRC channel, but he is left with the burden of proof that it works. ----------------------- -How do I compile PyPy? ----------------------- - -See the `getting-started`_ guide. - -.. _`getting-started`: getting-started-python.html -.. _`how do I compile my own interpreters`: +.. _compile-own-interpreters: -------------------------------------- How do I compile my own interpreters? ------------------------------------- + Begin by reading `Andrew Brown's tutorial`_ . -.. _`Andrew Brown's tutorial`: http://morepypy.blogspot.com/2011/04/tutorial-writing-interpreter-with-pypy.html +.. _Andrew Brown's tutorial: http://morepypy.blogspot.com/2011/04/tutorial-writing-interpreter-with-pypy.html + ---------------------------------------------------------- Can RPython modules for PyPy be translated independently? --------------------------------------------------------- @@ -202,7 +192,7 @@ undertaking. Consider it as quite unlikely for now.) .. _cffi: http://cffi.readthedocs.org/ --------------------------------------------------------------------- + Why does the translator draw a Mandelbrot fractal while translating? -------------------------------------------------------------------- diff --git a/rpython/doc/garbage_collection.rst b/rpython/doc/garbage_collection.rst index 89795aa7bd..68861b0417 100644 --- a/rpython/doc/garbage_collection.rst +++ b/rpython/doc/garbage_collection.rst @@ -1,3 +1,6 @@ +.. TODO cleanup after merge of gc-del + + ============================= Garbage Collection in RPython ============================= @@ -14,7 +17,7 @@ to that file for an old, but still more or less accurate, description. The present document describes the specific garbage collectors that we wrote in our framework. -.. _`EU-report on this topic`: https://bitbucket.org/pypy/extradoc/src/extradoc/eu-report/D07.1_Massive_Parallelism_and_Translation_Aspects-2007-02-28.pdf +.. _EU-report on this topic: https://bitbucket.org/pypy/extradoc/src/extradoc/eu-report/D07.1_Massive_Parallelism_and_Translation_Aspects-2007-02-28.pdf Garbage collectors currently written for the GC framework @@ -28,7 +31,7 @@ translation`_. The following overview is written in chronological order, so the "best" GC (which is the default when translating) is the last one below. -.. _`overview of command line options for translation`: config/commandline.html#translation +.. _overview of command line options for translation: config/commandline.html#translation Mark and Sweep -------------- diff --git a/rpython/doc/getting-started.rst b/rpython/doc/getting-started.rst index f86439be75..9cdf1ff35a 100644 --- a/rpython/doc/getting-started.rst +++ b/rpython/doc/getting-started.rst @@ -1,4 +1,3 @@ -============================ Getting Started with RPython ============================ @@ -43,11 +42,12 @@ A good introductory level articles are available: * `How to write interpreters in RPython`_ and `part 2`_ by Andrew Brown. -.. _`Fast Enough VMs in Fast Enough Time`: http://tratt.net/laurie/tech_articles/articles/fast_enough_vms_in_fast_enough_time +.. _Fast Enough VMs in Fast Enough Time: http://tratt.net/laurie/tech_articles/articles/fast_enough_vms_in_fast_enough_time + +.. _How to write interpreters in RPython: http://morepypy.blogspot.com/2011/04/tutorial-writing-interpreter-with-pypy.html -.. _`How to write interpreters in RPython`: http://morepypy.blogspot.com/2011/04/tutorial-writing-interpreter-with-pypy.html +.. _part 2: http://morepypy.blogspot.com/2011/04/tutorial-part-2-adding-jit.html -.. _`part 2`: http://morepypy.blogspot.com/2011/04/tutorial-part-2-adding-jit.html .. _try-out-the-translator: @@ -63,7 +63,7 @@ you need to download and install the usual (CPython) version of: * `Dot Graphviz`_ .. _Pygame: http://www.pygame.org/ -.. _`Dot Graphviz`: http://www.graphviz.org/ +.. _Dot Graphviz: http://www.graphviz.org/ To start the interactive translator shell do:: @@ -81,8 +81,9 @@ After that, the graph viewer pops up, that lets you interactively inspect the flow graph. To move around, click on something that you want to inspect. To get help about how to use it, press 'H'. To close it again, press 'Q'. + Trying out the type annotator -+++++++++++++++++++++++++++++ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ We have a type annotator that can completely infer types for functions like ``is_perfect_number`` (as well as for much larger examples):: @@ -94,7 +95,7 @@ Move the mouse over variable names (in red) to see their inferred types. Translating the flow graph to C code -++++++++++++++++++++++++++++++++++++ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The graph can be turned into C code:: @@ -111,8 +112,9 @@ version is now in a ``.so`` library. You can run it say using ctypes: >>> f(6) 1 + Translating the flow graph to CLI or JVM code -+++++++++++++++++++++++++++++++++++++++++++++ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ PyPy also contains a `CLI backend`_ and JVM backend which can translate flow graphs into .NET executables or a JVM jar @@ -150,12 +152,13 @@ users need the `.NET Framework SDK`_, while Linux and Mac users can use Mono_. To translate and run for the JVM you must have a JDK installed (at least version 6) and ``java``/``javac`` on your path. -.. _`CLI backend`: cli-backend.html -.. _`.NET Framework SDK`: http://msdn.microsoft.com/netframework/ +.. _CLI backend: cli-backend.html +.. _.NET Framework SDK: http://msdn.microsoft.com/netframework/ .. _Mono: http://www.mono-project.com/Main_Page + A slightly larger example -+++++++++++++++++++++++++ +~~~~~~~~~~~~~~~~~~~~~~~~~ There is a small-to-medium demo showing the translator and the annotator:: @@ -174,7 +177,7 @@ type simply:: Translating Full Programs -+++++++++++++++++++++++++ +~~~~~~~~~~~~~~~~~~~~~~~~~ To translate full RPython programs, there is the script ``translate.py`` in :source:`rpython/translator/goal`. Examples for this are a slightly changed version of @@ -185,8 +188,8 @@ Pystone:: This will produce the executable "targetrpystonedalone-c". The largest example of this process is to translate the `full Python -interpreter`_. There is also an FAQ about how to set up this process for `your -own interpreters`_. +interpreter`_. There is also an FAQ about how to set up this process for :ref:`your +own interpreters <compile-own-interpreters>`. There are several environment variables you can find useful while playing with the RPython: @@ -199,7 +202,6 @@ There are several environment variables you can find useful while playing with t By default RPython keeps only the last ``PYPY_USESSION_KEEP`` (defaults to 3) session dirs inside ``PYPY_USESSION_DIR``. Increase this value if you want to preserve C files longer (useful when producing lots of lldebug builds). -.. _`your own interpreters`: faq.html#how-do-i-compile-my-own-interpreters Sources ------- @@ -224,5 +226,5 @@ Sources use from rpython. -.. _`full Python interpreter`: http://pypy.readthedocs.org/en/latest/getting-started-python.html -.. _`RPython standard library`: rlib.html +.. _full Python interpreter: http://pypy.readthedocs.org/en/latest/getting-started-python.html +.. _RPython standard library: rlib.html diff --git a/rpython/doc/jit/index.rst b/rpython/doc/jit/index.rst index f2417b64d1..06d38e6bcb 100644 --- a/rpython/doc/jit/index.rst +++ b/rpython/doc/jit/index.rst @@ -1,6 +1,5 @@ -======================================================================== - JIT documentation -======================================================================== +JIT documentation +================= :abstract: @@ -15,16 +14,10 @@ Content ------------------------------------------------------------- +------- -- Overview_: motivating our approach +- :doc:`Overview <overview>`: motivating our approach -- Notes_ about the current work in PyPy +- :doc:`Notes <pyjitpl5>` about the current work in PyPy -- Hooks_ debugging facilities available to a python programmer - - -.. _Overview: overview.html -.. _Notes: pyjitpl5.html -.. TODO: Update link -.. _Hooks: ../jit-hooks.html +- :doc:`Hooks <pypy:jit-hooks>` debugging facilities available to a python programmer diff --git a/rpython/doc/jit/overview.rst b/rpython/doc/jit/overview.rst index 52ef15307b..ea54e5e32a 100644 --- a/rpython/doc/jit/overview.rst +++ b/rpython/doc/jit/overview.rst @@ -1,6 +1,5 @@ ------------------------------------------------------------------------- - Motivating JIT Compiler Generation ------------------------------------------------------------------------- +Motivating JIT Compiler Generation +================================== .. contents:: @@ -9,10 +8,10 @@ to Just-In-Time compiler generation. Motivation -======================================================================== +---------- Overview --------- +~~~~~~~~ Writing an interpreter for a complex dynamic language like Python is not a small task, especially if, for performance goals, we want to write a @@ -41,7 +40,7 @@ getting out of sync with the JIT compiler. The path we followed --------------------- +~~~~~~~~~~~~~~~~~~~~ Our previous incarnations of PyPy's JIT generator were based on partial evaluation. This is a well-known and much-researched topic, considered @@ -73,7 +72,7 @@ interpreter for full Python. Practical results ------------------ +~~~~~~~~~~~~~~~~~ The JIT compilers that we generate use some techniques that are not in widespread use so far, but they are not exactly new either. The point @@ -100,7 +99,7 @@ that this should be: Alternative approaches to improve speed -======================================================================== +--------------------------------------- +----------------------------------------------------------------------+ | :NOTE: | @@ -183,12 +182,11 @@ Finally, tracing JITs are now emerging for dynamic languages like JavaScript with TraceMonkey. The code generated by PyPy is very similar (but not hand-written) to the concepts of tracing JITs. +.. _JSR 292: http://jcp.org/en/jsr/detail?id=292 + Further reading -======================================================================== +--------------- -The description of the current RPython JIT generator is given in PyJitPl5_ +The description of the current RPython JIT generator is given in :doc:`PyJitPl5 <pyjitpl5>` (draft). - -.. _`JSR 292`: http://jcp.org/en/jsr/detail?id=292 -.. _PyJitPl5: pyjitpl5.html diff --git a/rpython/doc/jit/pyjitpl5.rst b/rpython/doc/jit/pyjitpl5.rst index d0f19a7a03..1e3c21b4fc 100644 --- a/rpython/doc/jit/pyjitpl5.rst +++ b/rpython/doc/jit/pyjitpl5.rst @@ -1,29 +1,25 @@ -========== - PyJitPl5 -========== +PyJitPl5 +======== This document describes the fifth generation of the RPython JIT generator. Implementation of the JIT -========================= +------------------------- -The JIT's `theory`_ is great in principle, but the actual code is a different +The JIT's :doc:`theory <overview>` is great in principle, but the actual code is a different story. This section tries to give a high level overview of how RPython's JIT is -implemented. It's helpful to have an understanding of how the `RPython translation -toolchain`_ works before digging into the sources. +implemented. It's helpful to have an understanding of how the :doc:`RPython translation +toolchain <../translation>` works before digging into the sources. Almost all JIT specific code is found in rpython/jit subdirectories. Translation time code is in the codewriter directory. The metainterp directory holds platform independent code including the the tracer and the optimizer. Code in the backend directory is responsible for generating machine code. -.. _`theory`: overview.html -.. _`RPython translation toolchain`: ../translation.html - JIT hints ---------- +~~~~~~~~~ To add a JIT to an interpreter, RPython only requires two hints to be added to the target interpreter. These are jit_merge_point and can_enter_jit. @@ -44,7 +40,7 @@ the location of the JIT hints. JIT Generation --------------- +~~~~~~~~~~~~~~ After the RTyping phase of translation, where high level Python operations are turned into low-level ones for the backend, the translation driver calls @@ -69,7 +65,7 @@ final binary, and interpreter is ready to use the runtime component of the JIT. Tracing -------- +~~~~~~~ Application code running on the JIT-enabled interpreter starts normally; it is interpreted on top of the usual evaluation loop. When an application loop is @@ -139,13 +135,13 @@ the optimized assembly can be run instead of the normal interpreter. Optimizations -------------- +~~~~~~~~~~~~~ The JIT employs several techniques, old and new, to make machine code run faster. Virtuals and Virtualizables -*************************** +""""""""""""""""""""""""""" A *virtual* value is an array, struct, or RPython level instance that is created during the loop and does not escape from it via calls or longevity past the @@ -161,14 +157,14 @@ in the machine code. Virtualizables, however, can escape from JIT controlled code. Other optimizations -******************* +""""""""""""""""""" Most of the JIT's optimizer is contained in the subdirectory ``metainterp/optimizeopt/``. Refer to it for more details. More resources -============== +-------------- More documentation about the current JIT is available as a first published article: diff --git a/rpython/doc/rffi.rst b/rpython/doc/rffi.rst index 4f626b2dd7..829eee9ec3 100644 --- a/rpython/doc/rffi.rst +++ b/rpython/doc/rffi.rst @@ -8,11 +8,12 @@ This document describes an FFI for RPython language, concentrating on low-level backends like C. It describes how to declare and call low-level (C) functions from RPython level. + Declaring low-level external function ------------------------------------- Declaring external C function in RPython is easy, but one needs to -remember that low level functions eat `low level types`_ (like +remember that low level functions eat :ref:`low level types <low-level-types>` (like lltype.Signed or lltype.Array) and memory management must be done by hand. To declare a function, we write:: @@ -39,23 +40,19 @@ parameter:: name, args, result, compilation_info=info ) -See cbuild_ for more info on ExternalCompilationInfo. - -.. _`low level types`: rtyper.html#low-level-type -.. _cbuild: https://bitbucket.org/pypy/pypy/src/tip/pypy/translator/tool/cbuild.py +See :source:`cbuild <rpython/translator/tool/cbuild.py>` for more info on ExternalCompilationInfo. Types ----- -In rffi_ there are various declared types for C-structures, like CCHARP +In :source:`rffi <rpython/rtyper/lltypesystem/rffi.py>` there are various declared types for C-structures, like CCHARP (char*), SIZE_T (size_t) and others. refer to file for details. Instances of non-primitive types must be alloced by hand, with call to lltype.malloc, and freed by lltype.free both with keyword argument flavor='raw'. There are several helpers like string -> char* converter, refer to the source for details. -.. _rffi: https://bitbucket.org/pypy/pypy/src/tip/pypy/rpython/lltypesystem/rffi.py Registering function as external -------------------------------- @@ -63,12 +60,10 @@ Registering function as external Once we provided low-level implementation of an external function, would be nice to wrap call to some library function (like os.open) with such a call. For this, there is a `register_external` routine, -located in `extfunc.py`_, which provides nice API for declaring such a +located in :source:`extfunc.py <rpython/rtyper/extfunc.py>`, which provides nice API for declaring such a functions, passing llimpl as an argument and eventually llfakeimpl as a fake low-level implementation for tests performed by an llinterp. -.. _`extfunc.py`: https://bitbucket.org/pypy/pypy/src/tip/pypy/rpython/extfunc.py - OO backends ----------- diff --git a/rpython/doc/rlib.rst b/rpython/doc/rlib.rst index 07c4b4fcf3..95adeb8a47 100644 --- a/rpython/doc/rlib.rst +++ b/rpython/doc/rlib.rst @@ -1,9 +1,6 @@ -================================ Generally Useful RPython Modules ================================ -.. _Python: http://www.python.org/dev/doc/maint24/ref/ref.html - .. contents:: @@ -16,7 +13,7 @@ to change at some point. Usually it is useful to look at the tests in listsort -======== +-------- The :source:`rpython/rlib/listsort.py` module contains an implementation of the timsort sorting algorithm (the sort method of lists is not RPython). To use it, subclass from the @@ -27,22 +24,20 @@ argument, which will be sorted in place when the ``sort`` method of the be sorted using the ``listsort`` module in one program, otherwise the annotator will be confused. + nonconst -======== +-------- -The :source:`rpython/rlib/nonconst.py` module is useful mostly for tests. The `flow object space`_ and -the `annotator`_ do quite some constant folding, which is sometimes not desired +The :source:`rpython/rlib/nonconst.py` module is useful mostly for tests. The :ref:`flow object space <flow-object-space>` and +the :ref:`annotator <annotator>` do quite some constant folding, which is sometimes not desired in a test. To prevent constant folding on a certain value, use the ``NonConst`` class. The constructor of ``NonConst`` takes an arbitrary value. The instance of ``NonConst`` will behave during annotation like that value, but no constant folding will happen. -.. _`flow object space`: objspace.html#the-flow-object-space -.. _`annotator`: translation.html#the-annotation-pass - objectmodel -=========== +----------- The :source:`rpython/rlib/objectmodel.py` module is a mixed bag of various functionality. Some of the more useful ones are: @@ -92,7 +87,7 @@ more useful ones are: rarithmetic -=========== +----------- The :source:`rpython/rlib/rarithmetic.py` module contains functionality to handle the small differences in the behaviour of arithmetic code in regular Python and RPython code. Most of @@ -100,7 +95,7 @@ them are already described in the :doc:`RPython description <rpython>`. rbigint -======= +------- The :source:`rpython/rlib/rbigint.py` module contains a full RPython implementation of the Python ``long`` type (which itself is not supported in RPython). The ``rbigint`` class contains @@ -114,15 +109,16 @@ to add two rbigint instances). rrandom -======= +------- The :source:`rpython/rlib/rrandom.py` module contains an implementation of the mersenne twister random number generator. It contains one class ``Random`` which most importantly has a ``random`` method which returns a pseudo-random floating point number between 0.0 and 1.0. + rsocket -======= +------- The :source:`rpython/rlib/rsocket.py` module contains an RPython implementation of the functionality of the socket standard library with a slightly different interface. The @@ -133,16 +129,17 @@ a hierarchy of Address classes, in a typical static-OO-programming style. streamio -======== +-------- The :source:`rpython/rlib/streamio.py` contains an RPython stream I/O implementation (which was started by Guido van Rossum as `sio.py`_ in the CPython sandbox as a prototype for the upcoming new file implementation in Python 3000). -.. _`sio.py`: http://svn.python.org/view/sandbox/trunk/sio/sio.py +.. _sio.py: http://svn.python.org/view/sandbox/trunk/sio/sio.py + unroll -====== +------ The :source:`rpython/rlib/unroll.py` module most importantly contains the function ``unrolling_iterable`` which wraps an iterator. Looping over the iterator in RPython code will not @@ -150,7 +147,7 @@ produce a loop in the resulting flow graph but will unroll the loop instead. parsing -======= +------- The :source:`rpython/rlib/parsing/` module is a still in-development module to generate tokenizers and parsers in RPython. It is still highly experimental and only really used by the @@ -162,7 +159,8 @@ The regular expressions are implemented using finite automatons. The parsing engine uses `packrat parsing`_, which has O(n) parsing time but is more powerful than LL(n) and LR(n) grammars. -.. _`packrat parsing`: http://pdos.csail.mit.edu/~baford/packrat/ +.. _packrat parsing: http://pdos.csail.mit.edu/~baford/packrat/ + Regular Expressions ------------------- @@ -206,7 +204,8 @@ function ``make_runner(s)`` in the ``rpython.rlib.parsing.regexparse`` module. returns a object with a ``recognize(input)`` method that returns True or False depending on whether ``input`` matches the string or not. -.. _`re`: http://docs.python.org/library/re.html +.. _re: http://docs.python.org/library/re.html + EBNF ---- @@ -288,6 +287,7 @@ the expression ``12 + 4 * 5`` is parsed into the following tree: "-1213950420" [shape=box,label="DECIMAL\l'5'"]; } + Parse Trees ----------- @@ -305,7 +305,7 @@ of the nonterminal and ``children`` which is a list of the children attributes. Visitors -++++++++ +~~~~~~~~ To write tree visitors for the parse trees that are RPython, there is a special baseclass ``RPythonVisitor`` in :source:`rpython/rlib/parsing/tree.py` to use. If your @@ -330,8 +330,9 @@ more complex ones. The syntax for these transformations is to enclose symbols in expansions of a nonterminal by [...], <...> or >...<. + [symbol_1 symbol_2 ... symbol_n] -++++++++++++++++++++++++++++++++ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ This will produce a transformer that completely removes the enclosed symbols from the tree. @@ -381,8 +382,9 @@ After transformation the tree has the "," nodes removed: "-1219325492" [shape=box,label="__0_A\n'A'"]; } + <symbol> -++++++++ +~~~~~~~~ This will replace the parent with symbol. Every expansion can contain at most one symbol that is enclosed by <...>, because the parent can only be replaced @@ -452,8 +454,9 @@ After transformation the tree looks like this: "-1214414868" [shape=box,label="__5_d\n'd'"]; } + >nonterminal_1 nonterminal_2 ... nonterminal_n< -+++++++++++++++++++++++++++++++++++++++++++++++ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ This replaces the nodes nonterminal_1 to nonterminal_n by their children. @@ -536,6 +539,7 @@ children: "-1213442100" [shape=box,label="DECIMAL\n'5'"]; } + Extensions to the EBNF grammar format ------------------------------------- @@ -644,5 +648,5 @@ looks like this: } -.. _`Prolog interpreter`: https://bitbucket.org/cfbolz/pyrolog/ -.. _`json format`: http://www.json.org +.. _Prolog interpreter: https://bitbucket.org/cfbolz/pyrolog/ +.. _json format: http://www.json.org diff --git a/rpython/doc/rpython.rst b/rpython/doc/rpython.rst index d78b620394..700b35fc7c 100644 --- a/rpython/doc/rpython.rst +++ b/rpython/doc/rpython.rst @@ -1,4 +1,3 @@ -======= RPython ======= @@ -58,8 +57,8 @@ Flow restrictions **exceptions** -+ fully supported -+ see below `Exception rules`_ for restrictions on exceptions raised by built-in operations + fully supported + see below `Exception rules`_ for restrictions on exceptions raised by built-in operations Object restrictions diff --git a/rpython/doc/rtyper.rst b/rpython/doc/rtyper.rst index 02730c5ebb..4a79685643 100644 --- a/rpython/doc/rtyper.rst +++ b/rpython/doc/rtyper.rst @@ -10,8 +10,8 @@ The RPython Typer lives in the directory :source:`rpython/rtyper/`. Overview -------- -The RPython Typer is the bridge between the Annotator_ and the low-level code -generators. The annotations of the Annotator_ are high-level, in the sense +The RPython Typer is the bridge between the :ref:`Annotator <annotator>` and the low-level code +generators. The annotations of the :ref:`Annotator <annotator>` are high-level, in the sense that they describe RPython types like lists or instances of user-defined classes. In general, though, to emit code we need to represent these high-level annotations in the low-level model of the target language; for C, @@ -61,7 +61,7 @@ of addition (or concatenation maybe?) it means. The process in more details --------------------------- -The RPython Typer has a structure similar to that of the Annotator_: both +The RPython Typer has a structure similar to that of the :ref:`Annotator <annotator>` both consider each block of the flow graphs in turn, and perform some analysis on each operation. In both cases the analysis of an operation depends on the annotations of its input arguments. This is reflected in the usage of the same @@ -79,7 +79,7 @@ flow graph, by replacing each operation with some low-level operations. In addition to replacing operations, the RTyper creates a ``concretetype`` attribute on all Variables and Constants in the flow graphs, which tells code generators which type to use for each of them. This attribute is a -`low-level type`_, as described below. +:ref:`low-level type <low-level-types>`, as described below. Representations @@ -123,7 +123,7 @@ called ``rtype_op_xxx()`` which define how each high-level operation ``op_xxx`` is turned into low-level operations. -.. _`low-level type`: +.. _low-level-types: Low-Level Types --------------- @@ -219,7 +219,7 @@ though. The next pages are a reference guide. Primitive Types -+++++++++++++++ +~~~~~~~~~~~~~~~ Signed a signed integer in one machine word (a ``long``, in C) @@ -242,7 +242,7 @@ Void Structure Types -+++++++++++++++ +~~~~~~~~~~~~~~~ Structure types are built as instances of ``rpython.rtyper.lltypesystem.lltype.Struct``:: @@ -274,7 +274,7 @@ GcStruct: the parent structure uses the same GC header as the substructure. Array Types -+++++++++++ +~~~~~~~~~~~ An array type is built as an instance of ``rpython.rtyper.lltypesystem.lltype.Array``:: @@ -300,7 +300,7 @@ and the length of the array is specified at this time. Pointer Types -+++++++++++++ +~~~~~~~~~~~~~ As in C, pointers provide the indirection needed to make a reference modifiable or sharable. Pointers can only point to a structure, an array, a function @@ -332,7 +332,7 @@ turn into static immortal non-GC'ed data. Function Types -++++++++++++++ +~~~~~~~~~~~~~~ The declaration:: @@ -354,7 +354,7 @@ function in a way that isn't fully specified now, but the following attributes The PyObject Type -+++++++++++++++++ +~~~~~~~~~~~~~~~~~ This is a special type, for compatibility with CPython: it stands for a structure compatible with PyObject. This is also a "container" type (thinking @@ -370,7 +370,7 @@ The testing implementation allows you to "create" PyObjects by calling Opaque Types -++++++++++++ +~~~~~~~~~~~~ Opaque types represent data implemented in a back-end specific way. This data cannot be inspected or manipulated. @@ -432,7 +432,7 @@ very-low-level snippets of code. See for example :source:`rpython/rtyper/rlist.py`. -.. _`oo type`: +.. _oo-type: Object Oriented Types --------------------- @@ -449,7 +449,7 @@ these types, for testing purposes. The target platform -+++++++++++++++++++ +~~~~~~~~~~~~~~~~~~~ There are plenty of object oriented languages and platforms around, each one with its own native features: they could be statically or @@ -464,7 +464,7 @@ aren't. Types and classes -+++++++++++++++++ +~~~~~~~~~~~~~~~~~ Most of the primitive types defined in *ootypesystem* are the very same of those found in *lltypesystem*: ``Bool``, ``Signed``, @@ -491,7 +491,7 @@ as Java, may need to use some sort of placeholder instead. Static vs. dynamic typing -+++++++++++++++++++++++++ +~~~~~~~~~~~~~~~~~~~~~~~~~ The target platform is assumed to be **statically typed**, i.e. the type of each object is known at compile time. @@ -515,8 +515,9 @@ simply ignore some of the operation such as ``ooupcast`` and ``oodowncast``. Backends that supports implicit upcasting, such as CLI and Java, can simply ignore only ``ooupcast``. + Object model -++++++++++++ +~~~~~~~~~~~~ The object model implemented by ootype is quite Java-like. The following is a list of key features of the ootype object model which @@ -545,10 +546,11 @@ have a direct correspondence in the Java or .NET object model: can be easily simulated by backends for platforms on which it is not a native feature; - - there is a set of `built-in types`_ offering standard features. + - there is a set of :ref:`built-in-types` offering standard features. + Exception handling -++++++++++++++++++ +~~~~~~~~~~~~~~~~~~ Since flow graphs are meant to be used also for very low level backends such as C, they are quite unstructured: this means that the @@ -562,10 +564,11 @@ it. In particular the exception instances are typed with the ``Instance`` type, so the usual inheritance exception hierarchy is preserved and the native way to catch exception should just work. -.. `built-in types`_ + +.. _built-in-types: Built-in types -++++++++++++++ +~~~~~~~~~~~~~~ It seems reasonable to assume high level platforms to provide built-in facilities for common types such as *lists* or *hashtables*. @@ -588,7 +591,6 @@ built-in types: - **DictItemsIterator**: a helper class for iterating over the elements of a ``Dict`` - Each of these types is a subtype of ``BuiltinADTType`` and has set of ADT (Abstract Data Type) methods (hence the name of the base class) for being manipulated. Examples of ADT methods are ``ll_length`` for @@ -604,8 +606,9 @@ As an alternative, backends can special-case the ADT types to map them directly to the native equivalent, translating the method names on-the-fly at compile time. + Generics -++++++++ +~~~~~~~~ Some target platforms offer native support for **generics**, i.e. classes that can be parametrized on types, not only values. For @@ -743,7 +746,7 @@ Exception handling: is called.) -.. _LLInterpreter: +.. _llinterpreter: The LLInterpreter ----------------- @@ -789,5 +792,3 @@ you have to set the ``type_system`` parameter to the string return ~x res = interpret(f, [3], type_system='ootype') assert res == ~3 - -.. _annotator: translation.html#the-annotation-pass diff --git a/rpython/doc/translation.rst b/rpython/doc/translation.rst index e227b962f6..51611f7df8 100644 --- a/rpython/doc/translation.rst +++ b/rpython/doc/translation.rst @@ -1,4 +1,3 @@ -===================== The RPython Toolchain ===================== @@ -16,8 +15,9 @@ If you are reading this document for the first time, the Overview_ is likely to be most useful, if you are trying to refresh your PyPy memory then the `How It Fits Together`_ is probably what you want. + Overview -======== +-------- The job of the translation toolchain is to translate RPython programs into an efficient version of that program for one of various target platforms, @@ -33,12 +33,14 @@ The choice of the target platform affects the process somewhat, but to start with we describe the process of translating an RPython program into C (which is the default and original target). -.. _`initialization time`: +.. _initialization-time: + +.. TODO edit information about flow object space The RPython translation toolchain never sees Python source code or syntax trees, but rather starts with the *code objects* that define the behaviour of the function objects one gives it as input. The -`bytecode evaluator`_ and the `Flow Object Space`_ work through these +`bytecode evaluator`_ and the :ref:`Flow Object Space <flow-object-space>` work through these code objects using `abstract interpretation`_ to produce a control flow graph (one per function): yet another representation of the source program, but one which is suitable for applying type inference @@ -55,15 +57,15 @@ steps (see also the figure below): 2. The Annotator_ performs a global analysis starting from an specified entry point to deduce type and other information about what each - variable can contain at run-time, building flow graphs using the `Flow - Object Space`_ as it encounters them. + variable can contain at run-time, building flow graphs using the + :ref:`Flow Object Space <flow-object-space>` as it encounters them. -3. The `RPython Typer`_ (or RTyper) uses the high-level information +3. :ref:`rpython-typer` (or RTyper) uses the high-level information inferred by the Annotator to turn the operations in the control flow graphs into low-level operations. 4. After RTyping there are two, rather different, `optional - transformations`_ which can be applied -- the "backend + transformations <optional-transformations>` which can be applied -- the "backend optimizations" which are intended to make the resulting program go faster, and the "stackless transform" which transforms the program into a form of continuation passing style which allows the @@ -94,19 +96,17 @@ The following figure gives a simplified overview (`PDF color version`_): .. image:: _static/translation-greyscale-small.png -.. _`PDF color version`: _static/translation.pdf -.. _`bytecode evaluator`: interpreter.html -.. _`abstract interpretation`: http://en.wikipedia.org/wiki/Abstract_interpretation -.. _`Flow Object Space`: objspace.html#the-flow-object-space -.. _`interactive interface`: getting-started-dev.html#try-out-the-translator +.. _PDF color version: _static/translation.pdf +.. _bytecode evaluator: interpreter.html +.. _abstract interpretation: http://en.wikipedia.org/wiki/Abstract_interpretation + -.. _`flow-model`: -.. _`control flow graphs`: +.. _flow-model: The Flow Model -============== +-------------- -The `Flow Object Space`_ is described in the `document +The :ref:`Flow Object Space <flow-object-space>` is described in the `document describing object spaces`_. Here we describe the data structures produced by it, which are the basic data structures of the translation process. @@ -264,13 +264,13 @@ the types and their attributes in some detail: static, pre-initialized, read-only version of that object. The flow graph should not attempt to actually mutate such Constants. -.. _`document describing object spaces`: objspace.html +.. _document describing object spaces: objspace.html .. _annotator: The Annotation Pass -=================== +------------------- We describe briefly below how a control flow graph can be "annotated" to discover the types of the objects. This annotation pass is a form of type @@ -313,9 +313,11 @@ All the ``SomeXxx`` instances are immutable. If the annotator needs to revise its belief about what a Variable can contain, it does so creating a new annotation, not mutating the existing one. +.. _EU report about translation: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D05.1_Publish_on_translating_a_very-high-level_description.pdf + Mutable Values and Containers ------------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Mutable objects need special treatment during annotation, because the annotation of contained values needs to be possibly updated to account @@ -329,8 +331,9 @@ reflown through the relevant parts of the flow graphs. * ``SomeDict`` stands for a homogeneous dictionary (i.e. all keys have the same ``SomeXxx`` annotation, and so have all values). + User-defined Classes and Instances ----------------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ``SomeInstance`` stands for an instance of the given class or any subclass of it. For each user-defined class seen by the annotator, we @@ -375,12 +378,12 @@ unrelated annotations if they are not used in a general way through the parent class. -.. _`RPython typer`: +.. _rpython-typer: The RPython Typer -================= +----------------- -https://bitbucket.org/pypy/pypy/src/default/pypy/rpython/ +:source:`rpython/rtyper/` The RTyper is the first place where the choice of backend makes a difference; as outlined above we are assuming that ANSI C is the target. @@ -406,12 +409,13 @@ for making this step explicit and isolated in a single place. After RTyping, the graphs only contain operations that already live on the level of the target language, which makes the job of the code generators much simpler. -For more detailed information, see the `documentation for the RTyper`_. +For more detailed information, see the :doc:`documentation for the RTyper <rtyper>`. + +.. _documentation for the RTyper: rtyper.html -.. _`documentation for the RTyper`: rtyper.html Example: Integer operations ---------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~~ Integer operations are make an easy example. Assume a graph containing the following operation:: @@ -436,26 +440,29 @@ course, the purpose of replacing the operation called ``add`` with ``int_add`` is that code generators no longer have to worry about what kind of addition (or concatenation maybe?) it means. -.. _`optional-transformations`: + +.. _optional-transformations: The Optional Transformations -============================ +---------------------------- Between RTyping and C source generation there are two optional transforms: the "backend optimizations" and the "stackless transform". See also `D07.1 Massive Parallelism and Translation Aspects`_ for further details. -.. _`Technical report`: -.. _`D07.1 Massive Parallelism and Translation Aspects`: https://bitbucket.org/pypy/extradoc/raw/ee3059291497/eu-report/D07.1_Massive_Parallelism_and_Translation_Aspects-2007-02-28.pdf +.. _Technical report: +.. _D07.1 Massive Parallelism and Translation Aspects: https://bitbucket.org/pypy/extradoc/raw/ee3059291497/eu-report/D07.1_Massive_Parallelism_and_Translation_Aspects-2007-02-28.pdf + Backend Optimizations ---------------------- +~~~~~~~~~~~~~~~~~~~~~ The point of the backend optimizations are to make the compiled program run faster. Compared to many parts of the PyPy translator, which are very unlike a traditional compiler, most of these will be fairly familiar to people who know how compilers work. + Function Inlining +++++++++++++++++ @@ -493,6 +500,7 @@ time a function is being inlined into another function, the size of the outer function is recalculated. This is done until the remaining functions all have a size greater than a predefined limit. + Malloc Removal ++++++++++++++ @@ -542,25 +550,10 @@ frame they have been allocated in. This proved not to really gain us any speed, so over time it was removed again. -The Stackless Transform ------------------------ - -The stackless transform converts functions into a form that knows how -to save the execution point and active variables into a heap structure -and resume execution at that point. This was used to implement -coroutines as an RPython-level feature, which in turn are used to -implement coroutines, greenlets and tasklets as an application -level feature for the Standard Interpreter. - -The stackless transformation has been deprecated and is no longer -available in trunk. It has been replaced with continulets_. - -.. _continulets: stackless.html - -.. _`preparing the graphs for source generation`: +.. _preparing the graphs for source generation: Preparation for Source Generation -================================= +--------------------------------- This, perhaps slightly vaguely named, stage is the most recent to appear as a separate step. Its job is to make the final implementation decisions before @@ -576,8 +569,9 @@ the C backend, this step does three things: source (this mapping of objects to names is sometimes referred to as the "low-level database"). + Making Exception Handling Explicit ----------------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ RPython code is free to use exceptions in much the same way as unrestricted Python, but the final result is a C program, and C has no concept of @@ -586,13 +580,12 @@ similar way to CPython: exceptions are indicated by special return values and the current exception is stored in a global data structure. In a sense the input to the exception transformer is a program in terms of the -lltypesystem_ with exceptions and the output is a program in terms of the bare +:term:`lltypesystem` with exceptions and the output is a program in terms of the bare lltypesystem. -.. _lltypesystem: glossary.html#lltypesystem Memory Management Details -------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~ As well as featuring exceptions, RPython is a garbage collected language; again, C is not. To square this circle, decisions about memory management @@ -601,10 +594,9 @@ freedom to change how to do it. There are three approaches implemented today: - reference counting (deprecated, too slow) - using the `Boehm-Demers-Weiser conservative garbage collector`_ - - using one of our custom `exact GCs implemented in RPython`_ + - using one of our custom :doc:`exact GCs implemented in RPython <garbage_collection>` -.. _`Boehm-Demers-Weiser conservative garbage collector`: http://www.hpl.hp.com/personal/Hans_Boehm/gc/ -.. _`exact GCs implemented in RPython`: garbage_collection.html +.. _Boehm-Demers-Weiser conservative garbage collector: http://www.hpl.hp.com/personal/Hans_Boehm/gc/ Almost all application-level Python code allocates objects at a very fast rate; this means that the memory management implementation is critical to the @@ -616,22 +608,20 @@ You can choose which garbage collection strategy to use with .. _C: .. _GenC: -.. _`c backend`: +.. _c backend: The C Back-End -============== +-------------- -https://bitbucket.org/pypy/pypy/src/default/pypy/translator/c/ +:source:`rpython/translator/c/` GenC is usually the most actively maintained backend -- everyone working on PyPy has a C compiler, for one thing -- and is usually where new features are implemented first. -.. _`EU report about translation`: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D05.1_Publish_on_translating_a_very-high-level_description.pdf - A Historical Note -================= +----------------- As this document has shown, the translation step is divided into more steps than one might at first expect. It is certainly divided into more @@ -644,28 +634,26 @@ separately has become clear. Other backends -============== +-------------- Use the :config:`translation.backend` option to choose which backend to use. - The Object-Oriented Backends ----------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The Object-Oriented backends target platforms that are less C-like and support -classes, instance etc. If such a platform is targeted, the `OO type system` is +classes, instance etc. If such a platform is targeted, the :ref:`OO type system <oo-type>` is used while rtyping. Of the OO backends, both gencli and genjava can translate the full Python interpreter. -.. _`oo type system`: rtyper.html#oo-type - .. mention that pretty much all these backends are done by volunteers? + GenCLI ++++++ -GenCLI_ targets the `Common Language Infrastructure`_, the most famous +:doc:`GenCLI <cli-backend>` targets the `Common Language Infrastructure`_, the most famous implementations of which are Microsoft's `.NET`_ and Mono_. It is the most advanced of the object oriented backends -- it can @@ -677,11 +665,11 @@ It is almost entirely the work of Antonio Cuni, who started this backend as part of his `Master's thesis`_, the Google's Summer of Code 2006 program and the Summer of PyPy program. -.. _`Common Language Infrastructure`: http://www.ecma-international.org/publications/standards/Ecma-335.htm -.. _`.NET`: http://www.microsoft.com/net/ +.. _Common Language Infrastructure: http://www.ecma-international.org/publications/standards/Ecma-335.htm +.. _.NET: http://www.microsoft.com/net/ .. _Mono: http://www.mono-project.com/ -.. _`Master's thesis`: http://buildbot.pypy.org/misc/Implementing%20Python%20in%20.NET.pdf -.. _GenCLI: cli-backend.html +.. _Master's thesis: http://buildbot.pypy.org/misc/Implementing%20Python%20in%20.NET.pdf + GenJVM ++++++ @@ -697,17 +685,17 @@ richards benchmarks. GenJVM is almost entirely the work of Niko Matsakis, who worked on it also as part of the Summer of PyPy program. + .. _extfunccalls: External Function Calls -======================= +----------------------- -The external function call approach is described in `rffi`_ documentation. +The external function call approach is described in :doc:`rffi <rffi>` documentation. -.. _`rffi`: rffi.html How It Fits Together -==================== +-------------------- As should be clear by now, the translation toolchain of PyPy is a flexible and complicated beast, formed from many separate components. @@ -723,7 +711,7 @@ various components. It makes for a nice presentation to say that after the annotator has finished the RTyper processes the graphs and then the exception handling is made explicit and so on, but it's not entirely true. For example, the RTyper inserts calls to many -`low-level helpers`_ which must first be annotated, and the GC +:term:`low-level helper`\ s which must first be annotated, and the GC transformer can use inlining (one of the `backend optimizations`_) of some of its small helper functions to improve performance. The following picture attempts to summarize the components involved in @@ -732,8 +720,6 @@ performing each step of the default translation process: .. image:: _static/translation-detail-0.9.png :align: center -.. _`low-level helpers`: glossary.html#low-level-helper - A component not mentioned before is the "MixLevelAnnotator"; it provides a convenient interface for a "late" (after RTyping) translation step to declare that it needs to be able to call each of a diff --git a/rpython/doc/windows.rst b/rpython/doc/windows.rst index 9450ba6138..56d0280b3e 100644 --- a/rpython/doc/windows.rst +++ b/rpython/doc/windows.rst @@ -1,6 +1,5 @@ -================== -RPython on Windows -================== +Translating on Windows +====================== RPython is supported on Windows platforms, starting with Windows 2000. The following text gives some hints about how to translate a interpreter @@ -13,8 +12,7 @@ Microsoft Visual Studio is preferred, but can also use the mingw32 port of gcc. Translating PyPy with Visual Studio ----------------------------------- -We routinely test the `RPython translation toolchain`_ using -Visual Studio 2008, Express +We routinely test :doc:`translation` using Visual Studio 2008, Express Edition. Other configurations may work as well. The translation scripts will set up the appropriate environment variables @@ -34,6 +32,7 @@ The compiler is all you need to build pypy-c, but it will miss some modules that relies on third-party libraries. See below how to get and build them. + Preparing Windows for the large build ------------------------------------- @@ -52,6 +51,7 @@ Then you need to execute:: on the pypy.exe file you compiled. + Installing external packages ---------------------------- @@ -62,8 +62,10 @@ directory of the pypy checkout. For example, if you installed pypy in directory is ``d:\pypy``. You must then set the INCLUDE, LIB and PATH (for DLLs) environment variables appropriately. + Abridged method (for -Ojit builds using Visual Studio 2008) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + Download the versions of all the external packages from https://bitbucket.org/pypy/pypy/downloads/local.zip @@ -75,6 +77,7 @@ Then expand it into the base directory (base_dir) and modify your environment to Now you should be good to go. Read on for more information. + The Boehm garbage collector ~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -92,6 +95,7 @@ prompt:: nmake -f NT_THREADS_MAKEFILE copy Release\gc.dll <somewhere in the PATH> + The zlib compression library ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -102,6 +106,7 @@ the base directory. Then compile:: nmake -f win32\Makefile.msc copy zlib1.dll <somewhere in the PATH>\zlib.dll + The bz2 compression library ~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -111,6 +116,7 @@ the base directory. Then compile:: cd bzip2-1.0.5 nmake -f makefile.msc + The sqlite3 database library ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -139,6 +145,7 @@ is actually enough for PyPy). Then, copy the file ``win32\bin\release\libexpat.dll`` somewhere in your PATH. + The OpenSSL library ~~~~~~~~~~~~~~~~~~~ @@ -153,6 +160,7 @@ in the base directory. Then compile:: ms\do_ms.bat nmake -f ms\nt.mak install + Using the mingw compiler ------------------------ @@ -165,6 +173,7 @@ You probably want to set the CPATH, LIBRARY_PATH, and PATH environment variables to the header files, lib or dlls, and dlls respectively of the locally installed packages if they are not in the mingw directory heirarchy. + libffi for the mingw compiler ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -200,6 +209,7 @@ If you wish to experiment with win64, you must run configure with flags:: or such, depending on your mingw64 download. + hacking on Pypy with the mingw compiler ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Since hacking on Pypy means running tests, you will need a way to specify @@ -207,8 +217,7 @@ the mingw compiler when hacking (as opposed to translating). As of March 2012, --cc is not a valid option for pytest.py. However if you set an environment variable CC to the compliter exe, testing will use it. -.. _`mingw32 build`: http://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win32/Automated%20Builds -.. _`mingw64 build`: http://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win64/Automated%20Builds -.. _`msys for mingw`: http://sourceforge.net/projects/mingw-w64/files/External%20binary%20packages%20%28Win64%20hosted%29/MSYS%20%2832-bit%29 -.. _`libffi source files`: http://sourceware.org/libffi/ -.. _`RPython translation toolchain`: translation.html +.. _mingw32 build: http://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win32/Automated%20Builds +.. _mingw64 build: http://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win64/Automated%20Builds +.. _msys for mingw: http://sourceforge.net/projects/mingw-w64/files/External%20binary%20packages%20%28Win64%20hosted%29/MSYS%20%2832-bit%29 +.. _libffi source files: http://sourceware.org/libffi/ |