Lambda the Ultimate - Python

Google Brain's Jax and Flax

Fri, 15 Jan 2021 13:59:26 +0000

Google's AI division, Google Brain, has two main products for deep learning: TensorFlow and Jax. While TensorFlow is best known, Jax can be thought of as a higher-level language for specifying deep learning algorithms while automatically eliding code that doesn't need to run as part of the model.

Jax evolved from Autograd, and is a combination of Autograd and XLA. Autograd "can automatically differentiate native Python and Numpy code. It can handle a large subset of Python's features, including loops, ifs, recursion and closures, and it can even take derivatives of derivatives of derivatives. It supports reverse-mode differentiation (a.k.a. backpropagation), which means it can efficiently take gradients of scalar-valued functions with respect to array-valued arguments, as well as forward-mode differentiation, and the two can be composed arbitrarily. The main intended application of Autograd is gradient-based optimization."

Flax is then built on top of Jax, and allows for easier customization of existing models.

What do you see as the future of domain specific languages for AI?

Transfer of pywer

Thu, 12 Jul 2018 17:15:46 +0000

Guido van Rossum is "resigning" from being the Python BDFL: "I'm basically giving myself a permanent vacation from being BDFL, and you all will be on
your own." Apparently running a language can be tiring... It will be interesting to see what happens next.

Coroutines with async and await syntax (Python 3.5)

Thu, 17 Sep 2015 16:08:01 +0000

With Python 3.5 released, the thing that drew my attention is the support for asynchronous programming through PEP 0492 -- Coroutines with async and await syntax, which added awaitable objects, coroutine functions, asynchronous iteration, and asynchronous context managers. I found the mailing list discussions (look both above and below) particularly helpful in understanding what exactly is going on.

Ancient use of generators

Thu, 17 Sep 2015 05:58:53 +0000

Guido van Rossum reminisces a bit about early discussions of generators in the Python community (read the other messages in the thread as well). I think we talked about the articles he mentions way back when. Earlier still, and beyond the discussion by Guido here, was Icon, a clever little language that I have a soft spot for. i don't think we ever fully assessed its influence on Python and other languages.

Python and Scientific Computing

Tue, 19 Nov 2013 01:50:34 +0000

This interesting blog post argues that in recent years Python has gained libraries making it the choice language for scientific computing (over MATLAB and R primarily).

I find the details discussed in the post interesting. Two small points that occur to me are that in several domains Mathematica is still the tool of choice. From what I could see nothing free, let alone open source, is even in the same ballpark in these cases. Second, I find it interesting that several of the people commenting mentioned IPython. It seems to be gaining ground as the primary environment many people use.

Project Sikuli

Sun, 24 Jan 2010 05:31:57 +0000

Picture or screenshot driven programming from the MIT.

From the Sikuli project page:

Sikuli is a visual technology to search and automate graphical user interfaces (GUI) using images (screenshots). The first release of Sikuli contains Sikuli Script, a visual scripting API for Jython, and Sikuli IDE, an integrated development environment for writing visual scripts with screenshots easily.

The Development of Sage

Fri, 11 Dec 2009 21:53:19 +0000

Sage is a project to create a viable free open source alternative to Magma, Maple, Mathematica and Matlab. The lead developer/manager William Stein has recently written Mathematical Software and Me: A Very Personal Recollection, a rather enjoyable story of his experience with mathematical software, especially Magma, and how Sage came to be.

One of the difficulties of writing broadly useful math software is the sheer size and scope of such a project. It is easily outside the abilities of even the most prodigious lone developer. So the focus of Sage, at least up until recently, has been on creating Python-based interfaces to existing mathematical software. For example, for symbolic calculation the Sage distribution includes Maxima (written in Common Lisp), a fork of Macsyma dating back to the early 1980s, and released as open-source software by the US Department of Energy approximately 10 years ago. In addition to Maxima, Sage includes the ability to call out to Magma, Mathematica, and Maple.

There are some interesting PLT-related snippets, for example, Magma's language is frequently criticized, although its algorithms are frequently praised. In conversations with others, OCaml and Haskell were brought up, but William Stein chose Python because he felt that it was more accessible. Also, Axiom, which includes the dependently-typed language Aldor, was rejected in favor of Maxima because Maxima was less esoteric and much more widely used.

Unladen Swallow: LLVM based Python compiler

Wed, 15 Jul 2009 04:29:25 +0000

The second release of Unladen Swallow was made available yesterday. The project plan describes the goals as follows:

We want to make Python faster, but we also want to make it easy for large, well-established applications to switch to Unladen Swallow.

Produce a version of Python at least 5x faster than CPython.

Python application performance should be stable.

Maintain source-level compatibility with CPython applications.

Maintain source-level compatibility with CPython extension modules.

We do not want to maintain a Python implementation forever; we view our work as a branch, not a fork.

Of course, Google is known for writing a lot of their software in Python, and now they are trying to speed it up, just like the V8 project has done for JavaScript.

Skipping C - SPE and synthetic programming in Python

Tue, 14 Aug 2007 08:11:16 +0000

Expression and Loop Libraries for High-Performance Code Synthesis. Christopher Mueller and Andrew Lumsdaine. LCPC2006.

To simultaneously provide rapid application development and high performance, developers of scientific and multimedia applications often mix languages, using scripting languages to glue together high-performance components written in compiled languages. While this can be a useful development strategy, it distances application developers from the optimization process while adding complexity to the development tool chain. Recently, we introduced the Synthetic Programming Environment (SPE) for Python, a library for generating and executing machine instructions at run-time.

The authors didn't show much interest yet in supporting the most widespread ISAs, those for Intel processors. Instead they focus on PowerPC but also Cell. Have fun with Python and PS3 hacking.

Edit: You might also checkout the site of Christopher Mueller containing related material.

Google Python Sprint 2007

Mon, 13 Aug 2007 21:37:28 +0000

Joining this event (August 22-25) or following the action from afar may be a good way to keep up with Python 3000.

The first alpha release (3.0a1) should be just around the corner.

Python 3000 Status Update

Wed, 20 Jun 2007 07:19:48 +0000

Guido updates the vision. Syntax focus seems cancerous, leaving FP sandwiched between set literals and backtick syntax, under Miscellany:

reduce() is gone. This doesn't mean I don't like higher-order functions; it simply reflects that almost all code that uses reduce() becomes more readable when rewritten using a plain old for-loop. (Example.)

lambda, however, lives.

Guido says that example "gives reduce() a bad name" which leaves me wondering as to its relevance. The idea that FP = Miscellany makes me wonder why the more stunning Python success stories (Google) involve FP techniques. Elsewhere:

zip(), map(), filter() return iterables

So FP is not even a free-standing category yet. I sometimes wish PLT Spy would revive, or that some FP language would target Python intermediate code. The value of Python is often stated to be its libraries.

I also wonder if Python people might build intuition by watching REBOL (=Lisp) people at work. They seem to enjoy puzzles. The motivating notion would be "hey! that REBOL trick should be so easy in Python."

Python in Pardus Linux

Tue, 19 Jun 2007 22:55:33 +0000

Pardus Linux is a case study of functional Python. It's a Linux distribution built from semi-scratch, the main focii being package management and init subsystems - places where C and shell script make poor sense. A funded group has finally tackled these issues.

A package management software deals a lot with sets, lists, and dependency graphs....We have extensively used functional operators (map, filter, reduce) and list comprehensions, even metaclasses are used in a few places.

Someone nudge Guido. Scheme or Oz might have been the better choice, but give them credit. They admit frankly to social acceptance issues.

Charming Python: Decorators make magic easy

Fri, 05 Jan 2007 14:40:42 +0000

Python made metaprogramming possible, but each Python version has added slightly different -- and not quite compatible -- wrinkles to the way you accomplish metaprogramming tricks. Playing with first-class function objects has long been around, as have techniques for peaking and poking at magic attributes. With version 2.2, Python grew a custom metaclass mechanism that went a long way, but at the cost of melting users' brains. More recently, with version 2.4, Python has grown "decorators," which are the newest -- and by far the most user-friendly way, so far -- to perform most metaprogramming.

While metaprogramming is inherently a bit confusing, I think this article could have a been a little clearer. Still, it's a nice highlevel introduction to decorators.

Type inference for Python

Tue, 30 May 2006 01:54:44 +0000

The subject of type inference for dynamically-checked languages came up in the Buried Treasure thread. A question was raised in that thread having to do with why static type inference in these languages is difficult. Since there's a nascent body of literature which addresses that question, here are a few links to articles and papers about type inference for Python.

A nice overview can be found in Localized Type Inference of Atomic Types in Python, a Master's thesis by Brett Cannon. The whole thesis is relevant, but for an overview of the issues, see Chapter 3, "Challenges of Inferring Types in Python". Chapter 4 summarizes previous attempts involving static inference in Python, including Psyco (previously on LtU) and Starkiller. The limitations of these attempts are briefly addressed.

Type inference solutions for Python invariably involve restrictions to make the problem tractable. The above paper focuses on "inferring atomic types in the local namespace". Another approach is described in Aggressive Type Inference, by John Aycock. Aycock makes an important observation:

Giving people a dynamically-typed language does not mean that they write dynamically-typed programs.

The article offers a type inference approach which exploits this observation. (If the meaning of the above quote isn't clear, I recommend reviewing our mammoth three-part thread on the subject, "Why type systems are interesting", part I, part II, and part III.)

The PyPy implementation of Python in Python (previously on LtU) uses a restricted subset of Python, called RPython, to implement parts of the language. RPython is sufficiently static to be able to support full-program type inference. It is not a "soft" inference approach, and is not designed to be used with ordinary Python programs. The paper Compiling dynamic language implementations covers the approach used for static analysis of RPython. The PyPy Coding Guide, starting at section 1.4 may also be useful.

(It may be interesting to note that the PyPy approach is very similar to that used previously for Scheme 48. The core of Scheme 48 is implemented in PreScheme, a subset of Scheme that supports full-program type inference.)

Finally, Guido van Rossum has a number of blog entries on the subject of adding optional static typing to Python:

If anyone knows of any other good treatments of type inference in Python or similar languages, please post links here.

EasyExtend - Python MetaProgramming

Mon, 22 May 2006 22:37:55 +0000

Just saw this announcement on Google groups / comp.lang.python.

EasyExtend is a constructive approach to extend the Python language using pure Python. EasyExtend is developed as a Python framework depending only on tools provided by the CPython interpreter suite ( compiler ) and standard library as well as some pieces of code borrowed from the PyPy project. Opposite to toolkits for writing extension modules in C ( or RPython in future ) the EasyExtend framework is dedicated to extend the language itself by adding new grammar rules and transformation of parse trees. Acting directly on the nodes of syntax trees makes EasyExtend safe and extensible. Moreover the parser and the transformations are considerably fast. While EasyExtend can obviously be used to define "little languages" embedded in Python it can also be used to create Python oriented tools like a code coverage tool based on code generation. For both use-cases examples will be provided.

You'll want to probably want to check out the examples.