User loginNavigation |
Google Brain's Jax and FlaxGoogle's AI division, Google Brain, has two main products for deep learning: TensorFlow and Jax. While TensorFlow is best known, Jax can be thought of as a higher-level language for specifying deep learning algorithms while automatically eliding code that doesn't need to run as part of the model. Jax evolved from Autograd, and is a combination of Autograd and XLA. Autograd "can automatically differentiate native Python and Numpy code. It can handle a large subset of Python's features, including loops, ifs, recursion and closures, and it can even take derivatives of derivatives of derivatives. It supports reverse-mode differentiation (a.k.a. backpropagation), which means it can efficiently take gradients of scalar-valued functions with respect to array-valued arguments, as well as forward-mode differentiation, and the two can be composed arbitrarily. The main intended application of Autograd is gradient-based optimization." Flax is then built on top of Jax, and allows for easier customization of existing models. What do you see as the future of domain specific languages for AI? By Z-Bo at 2021-01-15 13:59 | Implementation | Python | Scientific Programming | Software Engineering | login or register to post comments | other blogs | 63175 reads
Built to LastMar Hicks. Built to Last. Logic. Issue 11, "Care".
Recently, work on the history of technology has been becoming increasingly more sophisticated and moved beyond telling the story of impressive technology to trying to unravel the social, political, and economic forces that affected the development, deployment, and use of a wide range of technologies and technological systems. Luckily, this trend is beginning to manifest itself in studies of the history of programming languages. While not replacing the need for careful, deeply informed, studies of the internal intellectual forces affecting the development of programming languages, these studies add a sorely needed aspect to the stories we tell. Tackling the Awkward Squad for Reactive ProgrammingSam Van den Vonder, Thierry Renaux, Bjarno Oeyen, Joeri De Koster, Wolfgang De Meuter Reactive programming is a programming paradigm whereby programs are internally represented by a dependency graph, which is used to automatically (re)compute parts of a program whenever its input changes. In practice reactive programming can only be used for some parts of an application: a reactive program is usually embedded in an application that is still written in ordinary imperative languages such as JavaScript or Scala. In this paper we investigate this embedding and we distill “the awkward squad for reactive programming” as 3 concerns that are essential for real-world software development, but that do not fit within reactive programming. They are related to long lasting computations, side-effects, and the coordination between imperative and reactive code. To solve these issues we design a new programming model called the Actor-Reactor Model in which programs are split up in a number of actors and reactors. Actors and reactors enforce a strict separation of imperative and reactive code, and they can be composed via a number of composition operators that make use of data streams. We demonstrate the model via our own implementation in a language called Stella. The Simple Essence of Algebraic Subtyping: Principal Type Inference with Subtyping Made EasyThe Simple Essence of Algebraic Subtyping: Principal Type Inference with Subtyping Made Easy, Lionel Parreaux, ICFP 2020.
There's also an introductory blog post and an online demo. Stephen Dolan's Algebraic Subtyping (discussion) unexpectedly provided a solution to the problem of combining type inference and subtyping, but used somewhat heavy and unusual machinery. Now Lionel Parreaux shows that the system can be implemented in a very straightforward and pleasing way. Here's to hoping that it makes it into real languages! Applications of Blockchain to Programming Language TheoryLet's talk about Blockchain. Goal is to use this forum topic to highlight its usefulness to programming language theory and practice. If you're familiar with existing research efforts, please share them here. In addition, feel free to generate ideas for how Blockchain could improve languages and developer productivity. As one tasty example: Blockchain helps to formalize thinking about mutual knowledge and common knowledge, and potentially think about sharing intergalactic computing power through vast distributed computing fabrics. If we can design contracts in such a way that maximizes the usage of mutual knowledge while minimizing common knowledge to situations where you have to "prove your collateral", third-party transactions could eliminate a lot of back office burden. But, there might be benefits in other areas of computer science from such research, as well. Some language researchers, like Mark S. Miller, have always dreamed of Agoric and the Decades-Long Quest for Secure Smart Contracts. Some may also be aware that verification of smart contracts is an important research area, because of the notorious theft of purse via logic bug in an Ethereum smart contract. By Z-Bo at 2020-04-13 14:38 | Fun | Implementation | Semantics | 4 comments | other blogs | 103335 reads
Turnstile+: Dependent Type Systems as MacrosIn 2017, a team from Northeastern University released Turnstile, a framework for implementing propositionally typed languages in Racket; cf. naasking's story Type Systems as Macros. The system was really nice because it allowed type systems to be expressed in a manner similar to the way theoretical PL researchers would in a paper, and because it hooked into Racket's clean compiler backend. Now Stephen Chang, one of that team, together with new coauthors Michael Ballantyne, Usamilo Turner and William Bowman, have released a rewrite that they call Turnstile+, together with a POPL article, Dependent Type Systems as Macros. From that article's introduction:
The code is available at https://github.com/stchang/macrotypes. Histogram: You have to know the past to understand the present by Tomas PetricekHistogram: You have to know the past to understand the present by Tomas Petricek, University of Kent
Applied Category Theory - The Emerging Science of CompositionalityAn enjoyable 25-minute introductory talk: YOW! Lambda Jam 2019 - Ken Scambler - Applied Category Theory (slides)
Previously: Seven Sketches in Compositionality: An Invitation to Applied Category Theory. (via Brian McKenna) By Manuel J. Simoni at 2019-08-05 14:08 | Category Theory | Paradigms | Spotlight | 11 comments | other blogs | 97076 reads
Tensor Considered HarmfulTensor Considered Harmful, by Alexander Rush
Thanks to Edward Z. Yang for pointing me to this "Considered Harmful" position paper. By Z-Bo at 2019-06-27 14:26 | Critiques | Implementation | Teaching & Learning | 6 comments | other blogs | 78475 reads
Seven Sketches in Compositionality: An Invitation to Applied Category Theory
Seven Sketches in Compositionality: An Invitation to Applied Category Theory
2018 by Brendan Fong and David I. Spivak Category theory is becoming a central hub for all of pure mathematics. It is unmatched in its ability to organize and layer abstractions, to find commonalities between structures of all sorts, and to facilitate communication between different mathematical communities. But it has also been branching out into science, informatics, and industry. We believe that it has the potential to be a major cohesive force in the world, building rigorous bridges between disparate worlds, both theoretical and practical. The motto at MIT is mens et manus, Latin for mind and hand. We believe that category theory—and pure math in general—has stayed in the realm of mind for too long; it is ripe to be brought to hand.A very approachable but useful introduction to category theory. It avoids the Scylla and Charybdis of becoming incomprehensible after page 2 (as many academic texts do), and barely scratching the surface (as many popular texts do). By Andris Birkmanis at 2019-04-28 03:53 | Category Theory | Teaching & Learning | 2 comments | other blogs | 88628 reads
|
Browse archives
Active forum topics |
Recent comments
25 weeks 6 days ago
25 weeks 6 days ago
25 weeks 6 days ago
48 weeks 13 hours ago
1 year 1 day ago
1 year 1 week ago
1 year 1 week ago
1 year 4 weeks ago
1 year 9 weeks ago
1 year 9 weeks ago