안드로이드 OpenGL ES 2, GLSL 소개

GLSL ( OpenGL Shader Language )

안드로이드 2.2부터 지원되는 OpenGL ES 2.0에서 사용할 수 있는 쉐이더 언어(Shader Language)의 간단한 소개 및 예제 설명

OpenGL ES 2를 이용하여  화면에 무엇인가를 출력하고 한다면 반드시 Shader  프로그램을 사용해야 한다.

Shader 프로그램은 Vertex Shader, Fragment Shader로 구분된다.

Vertex Shader는 Transformation(변환)을 담당하고, Fragment Shader는 Drawing(색상연산)을 담당한다.

GLSL의 특징

그래픽 하드웨어(GPU)에서 실행되며 작은 코드로 작성되고 빠르다

개발자가 렌더링 파이프라인에 개입할 수 있는 수단이 된다.

C언어와 유사한 구조 및 문법체계를 가지고 있다.

Android 2.2 부터 지원된다.

Vertex Shader, Fragment (Pixel) Shader로 구분된다.

Vertex Shader는 주로 Lighting, Transform, Scale 등을 다루며 전달되는 정점마다 한번씩 수행되고 그래픽 요소는 변경할 수 없다.

Fragment Shader는 Texture, Shading 을 주로 다루며 인접한 Fragment(Pixel)에는 영향을 미치지 못한다. 픽셀당 한번씩 수행

   Fragment Shader에서 산출한 색상 값은 결국 프레임 버퍼나 텍스쳐 버퍼를 갱신하게 된다.

GLSL 가 다루는 주요한 3가지 영역

위치 변환, 수학함수 (Matrix, Vector, .... )

색상 연산, Texture Sampling

Small Utilities

Shader 와 호출측 프로그램이 교환하는 데이터는 크게 다음과 같이 2가지로 분류된다.

Uniform : Shader를 호출하는 프로그램 측에서 전달하며 주로 행렬이나 프로젝션이며 Vertex Shader, Fragment Shader 양측에서 사용될 수 있다. 양측에서 사용될 경우 동일한 타입으로 선언되어야 한다

Varying Varibles : Vertex Shader에서 Fragment Shader로 전달된다

Variable Types Allowed by GLSL

Class	Type	Description
Primitives	float, int, bool	Standard definitions apply
Vectors	int, ivec2, ivec3, ivec4, float, vec2, vec3, vec4 bool, bvec2, bvec3, bvec4	Float, int, and bools are "one-  dimensional Vectors." Boolean Vectors hold only bool values int their components.
Matrices	mat2, mat3, mat4	No Boolean matrices here.
Modifiers	highp(24bit), mediump(16), lowp(10)	highp is default.

기본형 변수는 상수로도 선언될 수 있다.

const float x=1.0.

C언어처럴 구조체도 사용될 수 있다.

제한 사항

GLSL 는 GPU에서 실행되기 때문에 많은 제한이 있다. uniform 변수의 수는 128개로 제한되며, 다중 루프도 그 깊이가 제한되기도 한다.

그러나 그러한 제한 사항을 하드웨어로부터 확인할 수는 없기 때문에 개발자는 이러한 제한 사항을 염두에 두고 GLSL프로그램을 가능한 적은 코드로 작성해야 한다.

OpenGL ES 2.0 Pipeline Structure

From : http://en.wikibooks.org/wiki/OpenGL_Programming/OpenGL_ES_Overview

1. Vertex Shader
   • Inputs: Attributes (vertex position and other per-vertex attributes such as texture positions through Vertex Arrays/Vertex Buffers), Samplers (Textures), Uniforms (Constants)
   • Outputs: gl_Position (in Clip Coordinates), gl_FrontFacing (auto-generated), gl_PointSize (for Point Sprites), user-defined Varyings (to Fragment Shader)
2. Primitive Assembly
   • Triangles/Lines/Point Sprites
   1. Clipping
   2. Perspective Division (results in Device Coordinates)
   3. Viewport Transformation (results in Window Coordinates)
3. Rasterization
   1. Culling
   2. Depth Offset
   3. Varying Interpolation
   4. Fragment Shader
      • Inputs: gl_FragCoord, gl_FrontFacing, gl_PointCoord, Samplers (Textures), Uniforms (Constants), interpolated Varyings (from Vertex Shader)
      • Output: gl_FragColor
4. Fragment Operations
   1. Scissor Test
   2. Stencil Test
   3. Depth Test
   4. Blending
   5. Dithering
5. Rendering Target
   • Drawable provided by window system (direct rendering to screen)
   • Framebuffer and attached Renderbuffers (acting as Color, Depth, or Stencil buffer) or attached Texture buffers

About Shaders In General

Shaders are small programs compiled to run on the GPU. The syntax is similar to C, but many restrictions apply. Inputs to a shader are called Attributes (for per-vertex/per-fragment inputs) and Uniforms(for constants for all vertices/fragments). User-defined outputs are called Varyings.

Setting up Vertex and Fragment Shaders includes passing the shaders (as a string containing GLSL) to the OpenGL ES API, compiling both shaders and linking them (during linking, input/output Varying correspondence is checked), and binding buffers to Uniforms and Varyings.

[edit]Vertex Shader

The Vertex Shader is called once for each input vertex. The main task of the Vertex Shader is to provide vertex positions for the following stages of the pipeline. Additionally, it can calculate further attributes that can be used as input for the Fragment Shader later. The most basic shader just takes vertex positions as input and directly assigns the input data to the gl_Position Varying. Typically, the shader does a multiplication with the modelview projection matrix (passed as a Uniform constant) to allow translation and rotation of input geometry as well as perspective projection, possibly passes texture coordinates, and calculates lighting parameters.

Additional user outputs typically include:

• Texture coordinates. These may be just passed through from input attributes for simple texturing but also might get generated or processed for implementing reflective surfaces and environment mapping or other effects such as dynamic texturing.
• Fog factor. For a fog effect, the distance of the primitive from the eye can be calculated in the vertex shader. The fragment shader can later fade out the fragment based on this value.
• Lighting parameters. Based on light source positions (passed as Uniform constants) and vertex normals (needed as additional per-vertex input), lighting parameters can be generated for the fragment shader.

Note that texture access via Samplers in vertex shaders is optional in OpenGL ES 2.0 and might not be supported on some devices.

[edit]Primitive Assembly

In the Primitive Assembly stage, several coordinate transformations are done:

• Clipping. Primitives lying outside the viewing volume are discarted, and primitives lying partially outside the view will be clipped. Varying outputs of the vertex shader get clipped, too.
• Perspective Division. The three main elements of gl_Position (x, y, z) are normalized to [-1.0...1.0] by division by the fourth vertex element w. The result is normalized device coordinates.
• Viewport Transformation. Coordinates are transformed to window coordinates by means of a linear transformation using the parameters set by glViewport() and glDepthRangef().

[edit]Rasterization

Rasterization is the process of creating a two-dimensional rasterized image from a primitive, i.e., calculating the set of fragments (pixels) for each primitive. For polygon rasterization, this includes the following steps.

• Culling. Polygons viewed from the back can be discarted if enabled using FrontFace() and CullFace().
• Depth Offset. A depth offset can be applied to polygon coordinates using PolygonOffset(). This can prevent Z-fighting for polygons that lie in the same plane.
• Varying Interpolation. Vertex shader Varying outputs and depth are interpolated when prepared as input for the Fragment Shader.

[edit]Fragment Shader

The Fragment Shader is called once for each primitive fragment (pixel). The main task of the Fragment Shader is to provide color values for each output fragment. The most basic Fragment Shader just assigns a constant value to its gl_FragColor output. Typically, the Fragment Shader does a texture lookup and implements lighting based on the lighting parameters the Vertex Shader computed previously.

[edit]Fragment Operations

• Scissor testing. If enabled using glEnable(GL_SCISSOR_TEST), only pixels in a specified rectangular region are drawn. Configure using glScissor().
• Stencil buffer testing. If enabled using glEnable(GL_STENCIL_TEST), pixels may be updated only when passing a test against the stencil buffer. Configure using glStencil*().
• Depth buffer testing. If enabled using glEnable(GL_DEPTH_TEST), pixels are only drawn if passing the depth buffer test, implementing hidden-surface removal. Configure using glDepthFunc(). A depth buffer needs to be available.
• Blending. If enabled using glEnable(GL_BLEND), pixels output by the Fragment Shader may be blended with pixel values already present in the output buffer. Configuration of blending is done using glBlend*().
• Dithering. If enabled using glEnable(GL_DITHER), dithering may be used to increase the perceived color depth. No further control of the dithering process is possible.
• Antialiasing. Using glEnable(GL_SAMPLE_COVERAGE), glEnable(GL_SAMPLE_ALPHA_TO_COVERAGE), and glSampleCoverage(), simple antialiasing may be configured.

[edit]Rendering Target

There are several possible targets for the generated pixels.

• Drawable provided by window system (direct rendering to screen)
• Framebuffer and attached Renderbuffers

Framebuffer objects have three attached objects: Depth, or Stencil Renderbuffer, and a Color renderbuffer or Texture Buffer. Color renderbuffers cannot be used as texture sources.

• glGenRenderbuffers(), glGenFramebuffers(), glBindRenderbuffer(), glRenderbufferStorage(), glBindFramebuffer(), glFramebufferRenderbuffer(), glFramebufferTexture*(), glCheckFramebufferStatus(), glDeleteRenderbuffers(), glDeleteFramebuffers()

Vertex Shader 프로그램의 예

uniform mat4 u_mvpMatrix;

attribute vec4 a_position;
attribute vec4 a_color;

varying vec4 v_color;

void main()
{
   gl_Position = u_mvpMatrix * a_position;
   v_color = a_color;
}

Fragment (Pixel) Shader 프로그램의 예 ( Fragment shaders compute the color of a fragment (pixel) )

varying vec4 v_color;

void main()
{
	gl_FragColor = v_color;
}

Shader 컴파일 예

아래의 createProgram()메소드는 GLSurfaceView.Render의 onSurfaceCreated()에서 호출하면 된다.

private int createProgram(String vertexSource, String fragmentSource) {                          //1

int vertexShader = loadShader(GLES20.GL_VERTEX_SHADER, vertexSource);                 //2

    if (vertexShader == 0) {

       return 0;

     }

int pixelShader = loadShader(GLES20.GL_FRAGMENT_SHADER, fragmentSource);

if (pixelShader == 0) {

    return 0;

}

int program = GLES20.glCreateProgram();                                                                  //3

if (program != 0) {

    GLES20.glAttachShader(program, vertexShader);                                                   //4

    GLES20.glAttachShader(program, pixelShader);

    GLES20.glLinkProgram(program);                                                                         //5

    int[] linkStatus = new int[1];

    GLES20.glGetProgramiv(program, GLES20.GL_LINK_STATUS, linkStatus, 0);             //6

    if (linkStatus[0] != GLES20.GL_TRUE) {

       Log.e(TAG, "Could not link program: ");

       Log.e(TAG, GLES20.glGetProgramInfoLog(program));

       GLES20.glDeleteProgram(program);

      program = 0;

   }

}

return program;                                                                                                     //7

}

private int loadShader(int shaderType, String source) {

int shader = GLES20.glCreateShader(shaderType);                                                   //8

if (shader != 0) {

    GLES20.glShaderSource(shader, source);                                                           //9

    GLES20.glCompileShader(shader);                                                                 //10

    int[] compiled = new int[1];

    GLES20.glGetShaderiv(shader, GLES20.GL_COMPILE_STATUS, compiled, 0);        //11

    if (compiled[0] == 0) {

        Log.e(TAG, "Could not compile shader " + shaderType + ":");

        Log.e(TAG, GLES20.glGetShaderInfoLog(shader));

        GLES20.glDeleteShader(shader);

        shader = 0;

     }

  }

return shader;                                                                                                  //12

}

Vertex Shader example (정점당 한번씩 호출된다)

uniform mat4 uMVPMatrix; // 호출측에서 4x4 행렬을 전달한다. 이곳에서 이 행렬을 사용하지 않으면 오류가 발생한다

attribute vec4 aPosition; // 호출측에서 전달하는 배열이며 모든 정점에 매핑될 데이터이다. 버텍스 쉐이더에서만  사용할 수 있다.

attribute vec2 aTextureCoord;

varying vec2 vTextureCoord // Fragment Shader로 전달되는 데이터

void main () 

{

    gl_Position = uMVPMatrix * aPosition; // gl_Position 은 시스템으로 전달되는 내장 변수

    vTextureCoord = aTextureCoord; // Framgment Shader(Pixel Shader)로 전달될 값을 할당함

}

Fragment Shader example

precision mediump float; // 쉐이더의 정밀도를 설정한다

varying vec2 vTextureCoord; // Vertex Shader에서 Fragment Shader로 전달되는 값. 양측의 변수명과 자료형이 일치해야 한다.

uniform sampler2D sTexture; // sampler1D, sampler2D, sampler3D등은 텍스쳐 정보를 Fragment Shader로 전달하기 위한 내장 타입

void main()

{

    gl_FragColor = texture2D(sTexture, vTextureCoord); // Fragment Shader에서 시스템으로 전달할  값을 할당한다

    // 개발자는 이 곳에서 다양한 계산을 수행할 수 있다

}

Fragment Shader에서 varying 변수는 Read-Only 상태로 사용되나, Vertex Shader에서는 그런 제한이 없다.

Shader를 사용하는 절차

위와 같은 쉐이더가 준비되어 있다면 사용할 절차만 남아 있다.

쉐이더에  선언된 uniform, attribute 변수는 호출 프로그램에서 쉐이더와 데이터 전달을 위해 사용되기 때문에 호출측 프로그램에서는 쉐이더의 uniform, attribute 변수의 핸들(Handle)을 구한 다음에 사용해야 한다.

Getting the Handles to the Uniforms and Attributes

public void onSurfaceCreated(GL10 glUnused, EGLConfig config)

{

   mProgram = createProgram(mVertexShader, mFragmentShader); //1

   if (mProgram == 0) {

      return;

   }

   maPositionHandle = GLES20.glGetAttribLocation(mProgram, "aPosition"); //2

   if (maPositionHandle == -1) {

      throw new RuntimeException("Could not get attrib location for aPosition");

   }

   maTextureHandle = GLES20.glGetAttribLocation(mProgram, "aTextureCoord"); //3

   if (maTextureHandle == -1) {

      throw new RuntimeException("Could not get attrib location for aTextureCoord");

   }

   muMVPMatrixHandle = GLES20.glGetUniformLocation(mProgram, "uMVPMatrix"); //4

      if (muMVPMatrixHandle == -1) {

        throw new RuntimeException("Could not get attrib location for uMVPMatrix");

      }

}

위에서 쉐이더의 변수(uniform, attribute타입)의 핸들(Handle)을 구했으므로 그 변수를 사용하면 된다

Calling and Using the Shaders

public void onDrawFrame(GL10 glUnused) //1

{

  GLES20.glClearColor(0.0f, 0.0f, 1.0f, 1.0f); //2

  GLES20.glClear( GLES20.GL_DEPTH_BUFFER_BIT | GLES20.GL_COLOR_BUFFER_BIT);

  GLES20.glUseProgram(mProgram); // 쉐이더 프로그램에 접근할 때는 항상 호출한다. 여러번 호출해도 된다

  GLES20.glActiveTexture(GLES20.GL_TEXTURE0); // Fragment Shader로 텍스쳐를 전달한다(sample2D)

  GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, mTextureID); //5

  mTriangleVertices.position(TRIANGLE_VERTICES_DATA_POS_OFFSET); //6

  GLES20.glVertexAttribPointer(maPositionHandle, 3, GLES20.GL_FLOAT, false, //7

  TRIANGLE_VERTICES_DATA_STRIDE_BYTES, mTriangleVertices);//정점정보를 쉐이더 변수에 전달한다

  GLES20.glEnableVertexAttribArray(maPositionHandle); //8

  mTriangleVertices.position(TRIANGLE_VERTICES_DATA_UV_OFFSET); //9

  GLES20.glVertexAttribPointer(maTextureHandle, 2, GLES20.GL_FLOAT, false,//10

  TRIANGLE_VERTICES_DATA_STRIDE_BYTES, mTriangleVertices);

  GLES20.glEnableVertexAttribArray(maTextureHandle); //11

  long time = SystemClock.uptimeMillis() % 4000L;

  float angle = 0.090f * ((int) time);

  Matrix.setRotateM(mMMatrix, 0, angle, 0, 0, 1.0f); //12 OpenGL ES2 에서는 glRotate(), glTranslate(), glScale()등이 지원 안됨

  Matrix.multiplyMM(mMVPMatrix, 0, mVMatrix, 0, mMMatrix, 0);

  Matrix.multiplyMM(mMVPMatrix, 0, mProjMatrix, 0, mMVPMatrix, 0);

  GLES20.glUniformMatrix4fv(muMVPMatrixHandle, 1, false, mMVPMatrix, 0); // 계산이 완료된 행렬을 Vertex Shader에 전달한다

  GLES20.glDrawArrays(GLES20.GL_TRIANGLES, 0, 3); //14

}