android, iphone, apps for iphone, iphone on verizon, app for android, iphone and v: Android RenderScript

RenderScript offers a high performance 3D graphics rendering and compute API at the native level, which you write in the C (C99 standard). The main advantages of RenderScript are:

Portability: RenderScript is designed to run on many types of devices with different CPU and GPU architectures. It supports all of these architectures without having to target each device, because the code is compiled and cached on the device at runtime.
Performance: RenderScript provides similar performance to OpenGL with the NDK while offering the portability of the OpenGL APIs provided by the Android framework (android.opengl). In addition, it also offers a high performance compute API that is not offered by OpenGL.
Usability: RenderScript simplifies development when possible, such as eliminating JNI glue code and simplifying mesh setup.

The main disadvantages are:

Development complexity: RenderScript introduces a new set of APIs that you have to learn. RenderScript also handles memory differently compared to OpenGL with the Android framework APIs or NDK.
Debugging visibility: RenderScript can potentially execute (planned feature for later releases) on processors other than the main CPU (such as the GPU), so if this occurs, debugging becomes more difficult.
Less features: RenderScript does not provide as many features as OpenGL such as all the compressed texture formats or GL extensions.

You need to consider all of the aspects of RenderScript before deciding when to use it. The following list describes general guidelines on when to use OpenGL (framework APIs or NDK) or RenderScript:

If you are doing simple graphics rendering and performance is not critical, you probably want to use the Android framework OpenGL APIs, which still provide adequate performance, to eliminate the added coding and debugging complexity of RenderScript.
If you want the most flexibility and features while maintaining relatively good debugging support, you probably want to use OpenGL and the NDK. Applications that require this are high end or complicated games, for example.
If you want a solution that is portable, has good performance, and you don't need the full feature set of OpenGL, RenderScript is a good solution. If you also need a high performance compute language, then RenderScript offers that as well. Good candidates for RenderScript are graphics intensive UIs that require 3D rendering, live wallpapers, or applications that require intensive mathematical computation.

For an example of RenderScript in action, install the RenderScript sample applications that are shipped with the SDK in<sdk_root>/samples/android-11/RenderScript. You can also see a typical use of RenderScript with the 3D carousel view in the Android 3.x versions of Google Books and YouTube.

RenderScript System Overview

The RenderScript system adopts a control and slave architecture where the low-level native code is controlled by the higher level Android system that runs in a virtual machine (VM). The Android VM still retains all control of memory and lifecycle management and calls the native RenderScript code when necessary. The native code is compiled to intermediate bytecode (LLVM) and packaged inside your application's .apk file. On the device, the bytecode is compiled (just-in-time) to machine code that is further optimized for the device that it is running on. The compiled code on the device is cached, so subsequent uses of the RenderScript enabled application do not recompile the intermediate code. RenderScript has three layers of code to enable communication between the native and Android framework code:

The native RenderScript layer does the intensive computation or graphics rendering. You define your native code in .rsand .rsh files.
The reflected layer is a set of classes that are reflected from the native code. It is basically a wrapper around the native code that allows the Android framework to interact with native RenderScripts. The Android build tools automatically generate the classes for this layer during the build process and eliminates the need to write JNI glue code, like with the NDK.
The Android framework layer is comprised of the Android framework APIs, which include the android.renderscriptpackage. This layer gives high level commands like, "rotate the view" or "filter the bitmap", by calling the reflected layer, which in turn calls the native layer.

Native RenderScript layer

The native RenderScript layer consists of your RenderScript code, which is compiled and executed in a compact and well defined runtime. Your RenderScript code has access to a limited amount of functions because it cannot access the NDK or standard C functions, since they must be guaranteed to run on a standard CPU. The RenderScript runtime was designed to run on different types of processors, which may not be the CPU, so it cannot guarantee support for standard C libraries. What RenderScript does offer is an API that supports intensive computation and graphics rendering with a collection of math and graphics APIs.

Some key features of the native RenderScript libraries include:

A large collection of math functions with both scalar and vector typed overloaded versions of many common routines. Operations such as adding, multiplying, dot product, and cross product are available.
Conversion routines for primitive data types and vectors, matrix routines, date and time routines, and graphics routines.
Logging functions
Graphics rendering functions
Memory allocation request features
Data types and structures to support the RenderScript system such as Vector types for defining two-, three-, or four-vectors.

The RenderScript header files and LLVM front-end libraries are located in the include and clang-include directories in the <sdk_root>/platforms/android-11/renderscript directory of the Android SDK. The headers are automatically included for you, except for the RenderScript graphics specific header file, which you can include as follows:

#include "rs_graphics.rsh"

Reflected layer

The reflected layer is a set of classes that the Android build tools generate to allow access to the native RenderScript code from the Android VM. This layer defines entry points for RenderScript functions and variables, so that you can interact with them with the Android framework. This layer also provides methods and constructors that allow you to allocate memory for pointers that are defined in your RenderScript code. The following list describes the major components that are reflected:

Every .rs file that you create is generated into a class named ScriptC_renderscript_filename of typeScriptC. This is the .java version of your .rs file, which you can call from the Android framework. This class contains the following reflections:
- Non-static functions in your .rs file.
- Non-static, global RenderScript variables. Accessor methods are generated for each variable, so you can read and write the natively declared variables from the Android framework. The get method comes with a one-way communication restriction. The last value that is set from the Android framework is always returned during a call to aget method. If the native RenderScript code changes the value, the change does not propagate back to the Android framework layer. If the global variables are initialized in the native RenderScript code, those values are used to initialize the corresponding values in the Android framework layer. If global variables are marked as const, then aset method is not generated.
- Global pointers generate a special method named bind_pointer_name instead of a set() method. This method allows you to bind the memory that is allocated in the Android VM for the pointer to the native RenderScript (you cannot allocate memory in your .rs file). You can read and write to this memory from both the Android framework and RenderScript code. For more information, see Working with Memory and Data
A struct is reflected into its own class named ScriptField_struct_name, which extends Script.FieldBase. This class represents an array of the struct, which allows you to allocate memory for one or more instances of thisstruct.

Android framework layer

The Android framework layer consists of the usual Android framework APIs, which include the RenderScript APIs inandroid.renderscript. This layer handles things such as the Activity lifecycle and memory management of your application. It issues high level commands to the native RenderScript code through the reflected layer and receives events from the user such as touch and input events and relays them to your RenderScript code, if needed.

Memory Allocation APIs

Before you begin writing your first RenderScript application, you must understand how memory is allocated for your RenderScript code and how data is shared between the native and VM spaces. RenderScript allows you to access allocated memory in both the native layer and Android system layer. All dynamic and static memory is allocated by the Android VM. The Android VM also does reference counting and garbage collection for you. You can also explicitly free memory that you no longer need.

Note: To declare temporary memory in your native RenderScript code without allocating it in the Android VM, you can still do things like instantiate a scratch buffer using an array.

The following classes support the memory management features of RenderScript in the Android VM. You normally do not need to work with these classes directly, because the reflected layer classes provide constructors and methods that set up the memory allocation for you. There are some situations where you would want to use these classes directly to allocate memory on your own, such as loading a bitmap from a resource or when you want to allocate memory for pointers to primitive types.

Android Object Type	Description
`Element`	An element represents one cell of a memory allocation and can have two forms: Basic or Complex. A basic element contains a single component of data of any valid RenderScript data type. Examples of basic element data types include a single float value, a float4 vector, or a single RGB-565 color. Complex elements contain a list of basic elements and are created from `struct`s that you declare in your RenderScript code. The most basic primitive type determines the data alignment of the memory. For example, a float4 vector subelement is alligned to `sizeof(float)` and not`sizeof(float4)`. The ordering of the elements in memory are the order in which they were added, with each component aligned as necessary.
`Type`	A type is a memory allocation template and consists of an element and one or more dimensions. It describes the layout of the memory (basically an array of `Element`s) but does not allocate the memory for the data that it describes. A type consists of five dimensions: X, Y, Z, LOD (level of detail), and Faces (of a cube map). You can assign the X,Y,Z dimensions to any positive integer value within the constraints of available memory. A single dimension allocation has an X dimension of greater than zero while the Y and Z dimensions are zero to indicate not present. For example, an allocation of x=10, y=1 is considered two dimensional and x=10, y=0 is considered one dimensional. The LOD and Faces dimensions are booleans to indicate present or not present.
`Allocation`	An allocation provides the memory for applications based on a description of the memory that is represented by a `Type`. Allocated memory can exist in many memory spaces concurrently. If memory is modified in one space, you must explicitly synchronize the memory, so that it is updated in all the other spaces that it exists in. Allocation data is uploaded in one of two primary ways: type checked and type unchecked. For simple arrays there are `copyFrom()` functions that take an array from the Android system and copy it to the native layer memory store. The unchecked variants allow the Android system to copy over arrays of structures because it does not support structures. For example, if there is an allocation that is an array of n floats, the data contained in a float[n] array or a byte[n*4] array can be copied.

Working with dynamic memory allocations

RenderScript has support for pointers, but you must allocate the memory in your Android framework code. When you declare a global pointer in your .rs file, you allocate memory through the appropriate reflected layer class and bind that memory to the native RenderScript layer. You can read and write to this memory from the Android framework layer as well as the RenderScript layer, which offers you the flexibility to modify variables in the most appropriate layer. The following sections show you how to work with pointers, allocate memory for them, and read and write to the memory.

Declaring pointers

Because RenderScript is written in C99, declaring a pointer is done in a familiar way. You can declare pointers to a structor a primitive type, but a struct cannot contain pointers or nested arrays. The following code declares a struct, a pointer to that struct, and a pointer of primitive type int32_t in an .rs file:

#pragma version(1)
#pragma rs java_package_name(com.example.renderscript)
...
typedef struct Point {
      float2 point;
  } Point_t;

  Point_t *touchPoints;
  int32_t *intPointer;
...

You cannot allocate memory for these pointers in your RenderScript code, but the Android build tools generate classes for you that allow you to allocate memory in the Android VM for use by your RenderScript code. These classes also let you read and write to the memory. The next section describes how these classes are generated through reflection.

How pointers are reflected

Global variables have a getter and setter method generated. A global pointer generates a bind_pointerName() method instead of a set() method. This method allows you to bind the memory that is allocated in the Android VM to the native RenderScript. For example, the two pointers in the previous section generate the following accessor methods in theScriptC_rs_filename file:


    private ScriptField_Point mExportVar_touchPoints;
    public void bind_touchPoints(ScriptField_Point v) {
        mExportVar_touchPoints = v;
        if (v == null) bindAllocation(null, mExportVarIdx_touchPoints);
        else bindAllocation(v.getAllocation(), mExportVarIdx_touchPoints);
    }

    public ScriptField_Point get_touchPoints() {
        return mExportVar_touchPoints;
    }

    private Allocation mExportVar_intPointer;
    public void bind_intPointer(Allocation v) {
        mExportVar_intPointer = v;
        if (v == null) bindAllocation(null, mExportVarIdx_intPointer);
        else bindAllocation(v, mExportVarIdx_intPointer);
    }

    public Allocation get_intPointer() {
        return mExportVar_intPointer;
    }

Allocating and binding memory to the RenderScript

When the build tools generate the reflected layer, you can use the appropriate class (ScriptField_Point, in our example) to allocate memory for a pointer. To do this, you call the constructor for the Script.FieldBase class and specify the amount of structures that you want to allocate memory for. To allocate memory for a primitive type pointer, you must build an allocation manually, using the memory management classes described in Table 1. The example below allocates memory for both the intPointer and touchPoints pointer and binds it to the RenderScript:

private RenderScriptGL glRenderer;
private ScriptC_example script;
private Resources resources;
public void init(RenderScriptGL rs, Resources res) {
   //get the rendering context and resources from the calling method
   glRenderer = rs; 
   resources = res; 
   
   //allocate memory for the struct pointer, calling the constructor
    ScriptField_Point touchPoints = new ScriptField_Point(glRenderer, 2); 
    
   //Create an element manually and allocate memory for the int pointer 
    intPointer = Allocation.createSized(glRenderer, Element.I32(glRenderer), 2); 
    
    //create an instance of the RenderScript, pointing it to the bytecode resource
    mScript = new ScriptC_example(glRenderer, resources, R.raw.example); 
    
    // bind the struct and int pointers to the RenderScript
    mScript.bind_touchPoints(touchPoints); 
    script.bind_intPointer(intPointer);
    
    //bind the RenderScript to the rendering context
    glRenderer.bindRootScript(script);
}

Reading and writing to memory

Although you have to allocate memory within the Android VM, you can work with the memory both in your native RenderScript code and in your Android code. Once memory is bound, the native RenderScript can read and write to the memory directly. You can also just use the accessor methods in the reflected classes to access the memory. If you modify memory in the Android framework, it gets automatically synchronized to the native layer. If you modify memory in the .rs file, these changes do not get propagated back to the Android framework. For example, you can modify the struct in your Android code like this:

int index = 0;
boolean copyNow = true;
Float2 point = new Float2(0.0f, 0.0f);
touchPoints.set_point(index, point, copyNow);

then read it in your native RenderScript code like this:

rsDebug("Printing out a Point", touchPoints[0].point.x, touchPoints[0].point.y);

Working with statically allocated memory

Non-static, global primitives and structs that you declare in your RenderScript are easier to work with, because the memory is statically allocated at compile time. Accessor methods to set and get these variables are generated when the Android build tools generate the reflected layer classes. You can get and set these variables using the provided accessor methods.

Note: The get method comes with a one-way communication restriction. The last value that is set from the Android framework is always returned during a call to a get method. If the native RenderScript code changes the value, the change does not propagate back to the Android framework layer. If the global variables are initialized in the native RenderScript code, those values are used to initialize the corresponding values in the Android framework layer. If global variables are marked as const, then a set method is not generated.

For example, if you declare the following primitive in your RenderScript code:

  uint32_t unsignedInteger = 1;

then the following code is generated in ScriptC_script_name.java:

 private final static int mExportVarIdx_unsignedInteger = 9;
    private long mExportVar_unsignedInteger;
    public void set_unsignedInteger(long v) {
        mExportVar_unsignedInteger = v;
        setVar(mExportVarIdx_unsignedInteger, v);
    }

    public long get_unsignedInteger() {
        return mExportVar_unsignedInteger;
    }

Note: The mExportVarIdx_unsignedInteger variable represents the index of the unsignedInteger's in an array of statically allocated primitives. You do not need to work with or be aware of this index.

For a struct, the Android build tools generate a class named<project_root>/gen/com/example/renderscript/ScriptField_struct_name. This class represents an array of the struct and allows you to allocate memory for a specified number of structs. This class defines:

Overloaded constructors that allow you to allocate memory. The ScriptField_struct_name(RenderScript rs, int count) constructor allows you to define the number of structures that you want to allocate memory for with thecount parameter. The ScriptField_struct_name(RenderScript rs, int count, int usages)constructor defines an extra parameter, usages, that lets you specify the memory space of this memory allocation. There are four memory space possibilities:
- USAGE_SCRIPT: Allocates in the script memory space. This is the default memory space if you do not specify a memory space.
- USAGE_GRAPHICS_TEXTURE: Allocates in the texture memory space of the GPU.
- USAGE_GRAPHICS_VERTEX: Allocates in the vertex memory space of the GPU.
- USAGE_GRAPHICS_CONSTANTS: Allocates in the constants memory space of the GPU that is used by the various program objects.
You can specify one or all of these memory spaces by OR'ing them together. Doing so notifies the RenderScript runtime that you intend on accessing the data in the specified memory spaces. The following example allocates memory for a custom data type in both the script and vertex memory spaces:
```
ScriptField_Point touchPoints = new ScriptField_Point(glRenderer, 2,
Allocation.USAGE_SCRIPT | Allocation.USAGE_GRAPHICS_VERTEX);
```
If you modify the memory in one memory space and want to push the updates to the rest of the memory spaces, callrsgAllocationSyncAll() in your RenderScript code to synchronize the memory.
A static nested class, Item, allows you to create an instance of the struct, in the form of an object. This is useful if it makes more sense to work with the struct in your Android code. When you are done manipulating the object, you can push the object to the allocated memory by calling set(Item i, int index, boolean copyNow) and setting theItem to the desired position in the array. The native RenderScript code automatically has access to the newly written memory.
Accessor methods to get and set the values of each field in a struct. Each of these accessor methods have an indexparameter to specify the struct in the array that you want to read or write to. Each setter method also has a copyNowparameter that specifies whether or not to immediately sync this memory to the native RenderScript layer. To sync any memory that has not been synced, call copyAll().
The createElement() method creates an object that describes the memory layout of the struct.
resize() works much like a realloc, allowing you to expand previously allocated memory, maintaining the current values that were previously set.
copyAll() synchronizes memory that was set on the framework level to the native level. When you call a set accessor method on a member, there is an optional copyNow boolean parameter that you can specify. Specifying truesynchronizes the memory when you call the method. If you specify false, you can call copyAll() once, and it synchronizes memory for the all the properties that are not synchronized.

The following example shows the reflected class, ScriptField_Point.java that is generated from the Point struct.

package com.example.renderscript;
import android.renderscript.*;
import android.content.res.Resources;

public class ScriptField_Point extends android.renderscript.Script.FieldBase {
    static public class Item {
        public static final int sizeof = 8;

        Float2 point;

        Item() {
            point = new Float2();
        }

    }

    private Item mItemArray[];
    private FieldPacker mIOBuffer;
    public static Element createElement(RenderScript rs) {
        Element.Builder eb = new Element.Builder(rs);
        eb.add(Element.F32_2(rs), "point");
        return eb.create();
    }

    public  ScriptField_Point(RenderScript rs, int count) {
        mItemArray = null;
        mIOBuffer = null;
        mElement = createElement(rs);
        init(rs, count);
    }

    public  ScriptField_Point(RenderScript rs, int count, int usages) {
        mItemArray = null;
        mIOBuffer = null;
        mElement = createElement(rs);
        init(rs, count, usages);
    }

    private void copyToArray(Item i, int index) {
        if (mIOBuffer == null) mIOBuffer = new FieldPacker(Item.sizeof * getType().getX()/* count */);
        mIOBuffer.reset(index * Item.sizeof);
        mIOBuffer.addF32(i.point);
    }

    public void set(Item i, int index, boolean copyNow) {
        if (mItemArray == null) mItemArray = new Item[getType().getX() /* count */];
        mItemArray[index] = i;
        if (copyNow)  {
            copyToArray(i, index);
            mAllocation.setFromFieldPacker(index, mIOBuffer);
        }

    }

    public Item get(int index) {
        if (mItemArray == null) return null;
        return mItemArray[index];
    }

    public void set_point(int index, Float2 v, boolean copyNow) {
        if (mIOBuffer == null) mIOBuffer = new FieldPacker(Item.sizeof * getType().getX()/* count */)fnati;
        if (mItemArray == null) mItemArray = new Item[getType().getX() /* count */];
        if (mItemArray[index] == null) mItemArray[index] = new Item();
        mItemArray[index].point = v;
        if (copyNow)  {
            mIOBuffer.reset(index * Item.sizeof);
            mIOBuffer.addF32(v);
            FieldPacker fp = new FieldPacker(8);
            fp.addF32(v);
            mAllocation.setFromFieldPacker(index, 0, fp);
        }

    }

    public Float2 get_point(int index) {
        if (mItemArray == null) return null;
        return mItemArray[index].point;
    }

    public void copyAll() {
        for (int ct = 0; ct < mItemArray.length; ct++) copyToArray(mItemArray[ct], ct);
        mAllocation.setFromFieldPacker(0, mIOBuffer);
    }

    public void resize(int newSize) {
        if (mItemArray != null)  {
            int oldSize = mItemArray.length;
            int copySize = Math.min(oldSize, newSize);
            if (newSize == oldSize) return;
            Item ni[] = new Item[newSize];
            System.arraycopy(mItemArray, 0, ni, 0, copySize);
            mItemArray = ni;
        }

        mAllocation.resize(newSize);
        if (mIOBuffer != null) mIOBuffer = new FieldPacker(Item.sizeof * getType().getX()/* count */);
    }
}

Disclaimer...

Android RenderScript