Home

Sh

From the University of Waterloo Computer Graphics Lab comes Sh:

Writing programs for these GPUs can be a tedious task, as it generally has to be done in assembly. A high-level language allows programming GPUs with familiar constructs and syntax, without worrying about the details of the hardware. A high-level language is also important for portability across different hardware and graphics API platforms. Sh is such a high-level language. It offers the convenient syntax of C++ and takes the burden of register allocation and other low-level issues away from the programmer. This allows GPU programs to be written much quicker and makes porting such programs extremely simple.

See also this recent article on Gamasutra (requires login, free)

By Dan Moniz at 2004-09-23 01:03 | DSL | other blogs | 11879 reads

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Sorry...

The convenient syntax of C++?

By mrd at Thu, 2004-09-23 04:44 | login or register to post comments

Sh Syntax

mrd: The convenient syntax of C++?

Absolutely, given that a game developer, for example, is almost certainly already working in C++, and the most common alternative is to write GPU assembly language for the GPU in question. The real question to ask is how Sh compares to other high-level approaches to programming the GPU, such as the Cg, GLSL, or DirectX Shader 2.0/3.0 languages.

By Paul Snively at Thu, 2004-09-23 05:45 | login or register to post comments

Comparisons

I agree, seeing some comparisons to Cg, HLSL, and OpenGL SL would be nice. A friend asked me the same question when I first posted this. The immediate obvious wins seem to be:

  • It's not tied to DirectX (HLSL is)
  • It's not as low level (syntactically) as Cg
  • It's not tied to NVIDIA hardware (Cg is)

Comparisons to OpenGL SL would be nice. From what limited reading I've done of Sh so far, it seems it's able to generate ARB fragment programs (as you would expect). I'll poke around some more and post any interesting divergences or similarities.

By Dan Moniz at Thu, 2004-09-23 19:00 | login or register to post comments

Cg and NVIDIA

Just to clarify: Cg is not tied to NVIDIA hardware: you can use the ARB_fp backend just as well as the NVIDIA-specific fp_{20,30}, for example. It was created by NVIDIA, but they had interoperability as a design goal.

I think it is the shader algebra part of Sh that makes it more interesting. Shaders are more first-class in Sh than in any other shading language. I'm not sure if there are any serious advantages of compositional design for vertex and fragment programs, especially since Sh has no automatic multipass capabilities. This pretty much rules it out in terms of performance when you're trying to create more complex shaders.

By Carlos Scheidegger at Fri, 2004-09-24 15:24 | login or register to post comments

Cg and NVIDIA

Carlos: thanks for the clarification, I didn't know this about Cg. Clearly I need to play with it more!

By Dan Moniz at Fri, 2004-09-24 19:33 | login or register to post comments

Shader Language Platform Independence

Here are a couple of other resources relative to using shader languages independent of any given runtime environment:

<http://www.graphics3d.com/guides/cg_1_1/index.html>

<http://xengine.sourceforge.net/features.php>

The point, of course, is that once you have an AST for the shader language, there's nothing to stop you from implementing it in terms of whatever runtime you can get to work, possibly even including OpenGL or DirectX on hardware without explicit support for vertex and/or pixel programs.

By Paul Snively at Fri, 2004-09-24 20:50 | login or register to post comments

One way to think of Sh is t

One way to think of Sh is that it's just a fancy API to a C++ library that represents an AST (well, an intermediate representation in the form of a flow graph, actually) and then outputs it in various forms. We actually eventually plan to target Cg, HLSL, etc. as backends, but one complication there is that those languages don't support goto, making it more difficult to target them from flow graphs. However, we're close to solving that problem now (we now target the latest NV assembly which has *structured* control flow, and NO goto, either...) so it should be possible to target Cg/HLSL/GLSL from Sh in the near future, but with the additional metaprogramming capabilities of Sh mentioned below.

One disadvantage of Sh worth mentioning: it's not a sandboxed special purpose language, it's a C++ API and library (no good noun for it; 'language' is not actually accurate). Downloading and running an arbitrary C++ program off a web page, for instance, would be Bad Idea. C++ is TOO powerful, and unfortunately it's not possible to bind Sh to Java (no operator overloading). It's therefore best to think of Sh as a toolkit for *building* shaders, and you may or may not want to use its JIT compilation and metaprogramming capabilities directly in a shipping application.

We also would like to bind Sh to a scripting language API in the future... after we do a lot of other things, and once we find (or create) a scripting language with the right properties: operator overloading, C syntax, objects, mechanism for defining custom control constructs, etc. That kind of mechanism would be a natural extension of scripting language support and usage in game engines, for instance, and would preserve the metaprogramming capabilities of Sh. Don't worry, we would definitely name such a binding something other than 'sh'!

Michael McCool, University of Waterloo

By mmccool at Wed, 2004-09-29 15:02 | login or register to post comments

The real question to ask is h

The real question to ask is how Sh compares to other high-level approaches to programming the GPU, such as the Cg, GLSL, or DirectX Shader 2.0/3.0 languages.

While these shading languages are almost all syntactic variants of one another, Sh takes a completely different approach to specifying shaders, because we don't add a whole new language to the mix but instead implemented Sh as a simple C++ library with a distinction between "retained" and "immediate" mode (a similar idea to OpenGL display lists).

This has many implications. The most obvious is that the shaders can be written directly within the application (and not just as an embedded text string). Variables can be shared with the application without having to include any binding code. This takes away almost all of the setup code involved in using a language like Cg.

Probably the most powerful implication is that all of the features of C++ are at your disposal: you can wrap shaders in classes, use templates or generative functions to generate variants of shaders and do all sorts of run-time code generation. As an example, I have written a function using Sh that, given a C++ string as input, returns a shader rendering that string (so we have a real "Hello World" shader!). This is only possible in other languages by writing a program that explicitly outputs code in the special shading language, using some other language.

We offer a lot more than that, for example our "Shader Algebra" (see the 2004 SIGGRAPH paper) features which allow you to recombine shaders in various ways. We also offer some optimizations that other languages don't (although to be fair we are missing some simple ones like common subexpression elimination).

And yes, there are some "non-technical" benefits associated with Sh: we're not strongly tied to any platform or vendor, and everything in Sh is completely open source (and free even for commercial use).

By sdt at Tue, 2004-09-28 21:10 | login or register to post comments

What an idiotic name

Sh the high-levek programming language as opposed to sh the ancient UNIX shell language.

What were the authors thinking?

By Charles A Stewart at Thu, 2004-09-23 08:13 | login or register to post comments

Consider the case...

We live in a case-sensitive world, man... Even verbally, that's "Sh", not "sh" -- the stress is on the first letter ;)

By sean at Thu, 2004-09-23 15:35 | login or register to post comments

What's in a Sh?

cas: Sh the high-levek [sic] programming language as opposed to sh the ancient UNIX shell language.

What were the authors thinking?

The authors are almost certainly too young to have "sh the ancient UNIX shell language" in their forebrains. Even if they currently use UNIX, they likely think of csh, tcsh, zsh, or bash rather than "sh." sh has essentially been relegated to shebang invocations for many years now.

By Paul Snively at Thu, 2004-09-23 16:03 | login or register to post comments

The authors are almost certai

The authors are almost certainly too young to have "sh the ancient UNIX shell language" in their forebrains. Even if they currently use UNIX, they likely think of csh, tcsh, zsh, or bash rather than "sh." sh has essentially been relegated to shebang invocations for many years now.

Well, we do know of sh the shell. The name Sh is sort of an unfortunate accident of history. I actually tried to change it but in the end we decided it was too late already (and yes, one of the reasons we don't think it's too huge a deal is that everyone uses a newer shell these days). While it is confusing sometimes, the confusion usually lasts only about 6 seconds, and the context more or less always gives away which Sh people are talking about, since they have very different target domains!

By sdt at Tue, 2004-09-28 21:02 | login or register to post comments

Vertigo

For a more functional approach to GPU programmig, see Conal Elliot's Vertigo, which was to be presented at this year's Haskell workshop (but unfortunately wasn't, because Conal couldn't make it). If you know Pan, it's similar, but instead it compiles a Haskell EDSL for 3D graphics directly to GPU code.

By Andre Pang at Sat, 2004-09-25 15:10 | login or register to post comments

Functional, Stream, Array, Metaprogramming

(disclaimer: I'm a researcher with the Sh project and an author of the book from AK Peters on it).

Just to generally reply to a bunch of comments at once: Sh has a lot of things that distinguish it from other approaches to programming GPUs.

  • First: it has a much stronger type system, since you can effectively use anything in C++. In combination with certain other properties of Sh (see data abstraction, below) this lets you create useful encapsulated implementations of various useful algorithms.
  • Second: Sh can target the host CPU (as well as the GPU), so you can use Sh to dynamically create functions. In fact, Sh functions create closures by capturing non-locally referenced parameters, so you effectively get a kind of lambda abstraction (however, Sh programs are not as general --- intentionally --- as arbitrary C++. See 'pure functions', below).
  • Third: Sh supports a syntax and a remote procedure abstraction that lets you use constructed program objects pretty much like functions, whether they run on the GPU or the CPU. This gives a simple single-threaded flow of control, even though the computation may in fact run in parallel on multiple processors.
  • Fourth: Our buffer management system lets you treat texture maps like arrays on either the host or on the GPU. There is also a stream abstraction for managing data with several channels, and a mechanism for automatically binding uniform parameters to shaders using the scope rules of C++ (creating closures when program objects are created; textures are treated the same way, as array-valued uniform parameters, roughly). Combining the above two points, Sh supports a stream programming model intended to permit general-purpose programming on the GPU. We are hard at work on the one hand to extend the semantics of this stream programming model to better support scientific computation (we're busy looking at languages like C* and APL to see what's useful), and on the other hand to support other backends (like parallel machines, not just GPUs). Another language worth comparing Sh to in this context is Brook. Sh can support BOTH general-purpose (well, scientific) computation and shader definition. Most other languages do one or the other, but not both. Our intent is to treat shaders as a special case of multi-value-return functions (which is what Sh program objects are).
  • Fifth: Sh is a metaprogramming language, so you can build up program objects dynamically. This makes it very easy to build 'custom compilers', effectively.
  • Sixth: Regarding a comparison with Vertigo and functional languages: program objects in Sh are essentially pure functions, with no side effects, even though they are specified internally using an imperative style. We feel this is a good compromise. The shader algebra basically adds a lightweight functional language capability to Sh: program objects can be functionally composed, inputs can be bound (with support for currying), functions can be applied to data objects, etc. This is done using two operators: << for functional composition and application, and & for code, I/O, and stream channel concatenation. Note that currying is implemented using just-in-time compilation and partial evaluation (where applicable, and curried inputs are marked as constants). We even support 'inverse currying': a bound parameter can be converted into an input using the >> operator.

I will freely admit that the implementation of Sh is currrently lacking some things, like multipass virtualization. Well, we're working on it; our goal is to get that working by the end of the year, but our manpower is limited.

Michael McCool
By mmccool at Wed, 2004-09-29 14:36 | login or register to post comments

Good Stuff!

Wow, thanks for the excellent outline of the benefits of Sh. I'd half-understoof some of them intuitively, but it's good to see an enumeration of them. I look forward to learning more about Sh just as soon as I get a machine with a programmable GPU. Now if that tax refund would just arrive... 15" PowerBook, here I come!

By Paul Snively at Sun, 2004-10-03 18:20 | login or register to post comments