Wednesday, August 05, 2009

gluUnProject for iPhone / OpenGL ES

I wasn't surprised that gluUnProject did not exist in the iPhone SDK since they did not provide a glu library implementation. However as I have migrated the gluLookAt, it was time to migrate the gluUnProject. It has been ages since I had used this method. The only thing I remebered was that it was used to know where you have clicked in your 3d world, by using a 2d device, i.e. a mouse. In our case since we're using iPhone, it's the capacitive touch screen which gives us back 2d coordinates of where we touched on the screen.

Since I'm using a perspective view, I needed to translate those coordinates into world coordinates. In my case, on the current project I'm working on, I would finally want to know which button the user click from a grid.

I got the gluUnProject code from MESA. However the code needed some minor adjustments, namely converting everything from double to float:
  • any GLdouble had to be replaced with GLfloat
  • any double numbers, e.g. 0.0 or 1.0 I converted them to their respective float counterpart, e.g. 0.0f or 1.0f
  • and math functions which accepted/returned double, I replaced them with their float versions, e.g. fabs -> fabsf
The next problem was that gluUnProject takes the 3 coordinates, 2 of them are retrieved from the touch event, but the Z coordinate we don't know it. On the desktop openGL usually the depth buffer is queried to retrieve the depth value (between 0 and 1) at the x-y coordinate. But we cannot do this on the iPhone as it uses tile rendering.

So I thought to use the value of zero, but in effect it was always giving the coordinates of the center of the screen. In reality it was giving me the coordinates of the camera position. When I tried 1 instead of 0, it was giving me coordinates that made more sense but still not 100% precise.

The solution was to unproject twice, one time at the near plane (z = 0) and one time at the far plane (z = 1) as discussed in a forum. That gives you a ray which can then be used to make an intersection with a plane and get the exact coordinates. Thus converting from 2d to 3d.

Here's the method I have used which uses the migrated gluUnProject using just floats:

-(CGPoint) getOGLPos:(CGPoint)winPos
{
// I am doing this once at the beginning when I set the perspective view
// glGetFloatv( GL_MODELVIEW_MATRIX, __modelview );
// glGetFloatv( GL_PROJECTION_MATRIX, __projection );
// glGetIntegerv( GL_VIEWPORT, __viewport );

//opengl 0,0 is at the bottom not at the top
winPos.y = (float)__viewport[3] - winPos.y;
// float winZ;
//we cannot do the following in openGL ES due to tile rendering
// glReadPixels( (int)winPos.x, (int)winPos.y, 1, 1, GL_DEPTH_COMPONENT24_OES, GL_FLOAT, &winZ );

float cX, cY, cZ, fX, fY, fZ;
//gives us camera position (near plan)
gluUnProject( winPos.x, winPos.y, 0, __modelview, __projection, __viewport, &cX, &cY, &cZ);
//far plane
gluUnProject( winPos.x, winPos.y, 1, __modelview, __projection, __viewport, &fX, &fY, &fZ);

//We could use some vector3d class, but this will do fine for now
//ray
fX -= cX;
fY -= cY;
fZ -= cZ;
float rayLength = sqrtf(cX*cX + cY*cY + cZ*cZ);
//normalize
fX /= rayLength;
fY /= rayLength;
fZ /= rayLength;

//T = [planeNormal.(pointOnPlane - rayOrigin)]/planeNormal.rayDirection;
//pointInPlane = rayOrigin + (rayDirection * T);

float dot1, dot2;

float pointInPlaneX = 0;
float pointInPlaneY = 0;
float pointInPlaneZ = 0;
float planeNormalX = 0;
float planeNormalY = 0;
float planeNormalZ = -1;

pointInPlaneX -= cX;
pointInPlaneY -= cY;
pointInPlaneZ -= cZ;

dot1 = (planeNormalX * pointInPlaneX) + (planeNormalY * pointInPlaneY) + (planeNormalZ * pointInPlaneZ);
dot2 = (planeNormalX * fX) + (planeNormalY * fY) + (planeNormalZ * fZ);

float t = dot1/dot2;

fX *= t;
fY *= t;
//we don't need the z coordinate in my case

return CGPointMake(fX + cX, fY + cY);
}

31 comments:

reacting said...

Can you post your conversion of gluUnProject?

Paul said...

Hi, I've converted the gluUnProject (I hope) and I'm trying to use your code to work out the actual coordinates in the 3D world, but the result I get back is 0,0. Any thoughts where I should check first, is it likely to be my conversion of UnProject?

Thanks for any help Paul

Unknown said...

I am having some problem getting valid gluUnProject point values as I am also implementing a glLookAt function. Anybody faced similar problems?

Paul said...

llias - my problems were firstly in my conversion and finally in the way I was saving the viewport data. My app allows the user to move the view around and herefore the viewport can change, I moved the storing of the viewport into the main draw code and all works ok

nonyabiz said...

man this conversion tough i am going use it for my custom term papers it is going to be tough explaining this thanks for this blog

Unknown said...

For gluUnProject, have a look at http://code.google.com/p/iphone-glu/ (not that I've personally tested this yet, but there's a function in there!)

s. p. stanley said...

Question: Do you get the model view matrix after you perform your gluLookAt()?

Unknown said...

Hi Xenon!

It would be possible that you post a download link to the modified libGlu.a and libGLU_arm.a?

I tried to compile by myself but I really don't know how to do it.

Really thanks for your post.

Regards.

Unknown said...

Hi Xenon!

Finally I managed to compile the modified GLU library and use your code to convert the screen coordinates to world coordinates.

Thanks for your post again!

Unknown said...

Maybe you can use a ray between camera position and only one point from glUnProject with zfar.

daniel john said...

There are certainly a lot of details like that to take into consideration. That is a great point to bring up. I offer the thoughts above as general inspiration but clearly there are questions like the one you bring up where the most important thing will be working in honest good faith. I don?t know if best practices have emerged around things like that, but I am sure that your job is clearly identified as a fair game.

Term papers

Ophir Paz said...

I used this sample to build a working sample for iPhone implementing gluUnproject :

http://blog.nova-box.com/2010/05/iphone-ray-picking-glunproject-sample.html#comment-form

Unknown said...

I think the interesting part of the project can be found in the glu.h / glu.c files and EAGLView.m.

College Term Paper

Unknown said...

iPhone is really the best.

PMP

Unknown said...

The gluUnProject for iPhone / OpenGL ES is really one that looks so much good, I think that the given information about it is really good one to see and has so much to get know about.

College Term Papers

Unknown said...

I always like The I-phone, And gluUn Project for i Phone is really good one for the Users of I-phone. And The Information is really great. I am thinking to forward it to my brother.

College Term Papers

Unknown said...

Thanks for doing such a thorough research work. I must say all the tips you have mentioned here are useful. So, I have saved the page. Hope it will help me in my future endeavors.

College Term Papers

sowmyaHere said...

hi ,


i got a question regarding multiple objects drawn using push/ pop stack.something similar to this.


glMatrixMode(GL_MODELVIEW);

glLoadIdentity();

glTranslatef(x,y,z);

glRotate(r,1,0,0);

glTranslate(-x,-y,-z);

for (i=0 to 20) objects

glpushMatrix();

draw_object

glpopMatrix();

end



each object is a unit cube (textured to sphere) with its own transformation.

In such a case how does ray picking work. How should i keep track of center point of the object to calculate ray intersection. i really appreciate any help.



Thanks


Sowmya

Unknown said...

Incidentally, I like the way you have structured your site, it is super and very easy to follow. I have bookmarked you and will be back regularly. Thank you

Study Consultants | Study Agent

Spiritual gifts said...

Thannk you for providing a thourough explanation for OpenGL. I am trying to fix a glitch that stops it from my iPhone.-spiritual gifts

Anonymous said...

I really agree with the facts that you have shared on this post. An interesting topic like this really enhances reader's mind to have more effective decisions over a certain issue.

CCTV Karachi

Term Papers said...

College term papers
No dispute with the text, agree with you. The features are really appreciable.

Atta Rehman said...

Valuable information and excellent design you got here! I would like to thank you for sharing your thoughts and time into the stuff you post!! Thumbs up

karachi online shopping

Jacob said...

Unproject twice to create a ray, ofcourse!!! Thanks =)

I creating a top down rpg, and want to click where the player is to move. So I create the ray, then find X and Y where Z = 0. Works great =)

Arsalan said...

This is a fantastic feature, other features are really fabulous though.


Custom written term papers

Sabra Allsbrook said...

Beautiful job! I just think they are brilliant. Thanks for this nice post.

cheap hotels in kansas city mo

Melany Flemmings said...

Interesting post. It is so awesome seeing young people love the world. Thanks for your thought and effort.

carry on luggage sizes

Crackshot69 said...

what would the code be to get the Z coordinate?

Crackshot69 said...

what would the code be to get the Z coordinate?

Unknown said...

I never read such beautiful article before. This article is very unique and full of surprise. I never read this kind of article that is straightforward when regarding to the topic. Very intense kind of writing. Keep up the good and unique article writing.

business plan service business plan writing service

Unknown said...

Can I have the same code for android.???