Wednesday, August 05, 2009

gluUnProject for iPhone / OpenGL ES

I wasn't surprised that gluUnProject did not exist in the iPhone SDK since they did not provide a glu library implementation. However as I have migrated the gluLookAt, it was time to migrate the gluUnProject. It has been ages since I had used this method. The only thing I remebered was that it was used to know where you have clicked in your 3d world, by using a 2d device, i.e. a mouse. In our case since we're using iPhone, it's the capacitive touch screen which gives us back 2d coordinates of where we touched on the screen.

Since I'm using a perspective view, I needed to translate those coordinates into world coordinates. In my case, on the current project I'm working on, I would finally want to know which button the user click from a grid.

I got the gluUnProject code from MESA. However the code needed some minor adjustments, namely converting everything from double to float:
  • any GLdouble had to be replaced with GLfloat
  • any double numbers, e.g. 0.0 or 1.0 I converted them to their respective float counterpart, e.g. 0.0f or 1.0f
  • and math functions which accepted/returned double, I replaced them with their float versions, e.g. fabs -> fabsf
The next problem was that gluUnProject takes the 3 coordinates, 2 of them are retrieved from the touch event, but the Z coordinate we don't know it. On the desktop openGL usually the depth buffer is queried to retrieve the depth value (between 0 and 1) at the x-y coordinate. But we cannot do this on the iPhone as it uses tile rendering.

So I thought to use the value of zero, but in effect it was always giving the coordinates of the center of the screen. In reality it was giving me the coordinates of the camera position. When I tried 1 instead of 0, it was giving me coordinates that made more sense but still not 100% precise.

The solution was to unproject twice, one time at the near plane (z = 0) and one time at the far plane (z = 1) as discussed in a forum. That gives you a ray which can then be used to make an intersection with a plane and get the exact coordinates. Thus converting from 2d to 3d.

Here's the method I have used which uses the migrated gluUnProject using just floats:

-(CGPoint) getOGLPos:(CGPoint)winPos
// I am doing this once at the beginning when I set the perspective view
// glGetFloatv( GL_MODELVIEW_MATRIX, __modelview );
// glGetFloatv( GL_PROJECTION_MATRIX, __projection );
// glGetIntegerv( GL_VIEWPORT, __viewport );

//opengl 0,0 is at the bottom not at the top
winPos.y = (float)__viewport[3] - winPos.y;
// float winZ;
//we cannot do the following in openGL ES due to tile rendering
// glReadPixels( (int)winPos.x, (int)winPos.y, 1, 1, GL_DEPTH_COMPONENT24_OES, GL_FLOAT, &winZ );

float cX, cY, cZ, fX, fY, fZ;
//gives us camera position (near plan)
gluUnProject( winPos.x, winPos.y, 0, __modelview, __projection, __viewport, &cX, &cY, &cZ);
//far plane
gluUnProject( winPos.x, winPos.y, 1, __modelview, __projection, __viewport, &fX, &fY, &fZ);

//We could use some vector3d class, but this will do fine for now
fX -= cX;
fY -= cY;
fZ -= cZ;
float rayLength = sqrtf(cX*cX + cY*cY + cZ*cZ);
fX /= rayLength;
fY /= rayLength;
fZ /= rayLength;

//T = [planeNormal.(pointOnPlane - rayOrigin)]/planeNormal.rayDirection;
//pointInPlane = rayOrigin + (rayDirection * T);

float dot1, dot2;

float pointInPlaneX = 0;
float pointInPlaneY = 0;
float pointInPlaneZ = 0;
float planeNormalX = 0;
float planeNormalY = 0;
float planeNormalZ = -1;

pointInPlaneX -= cX;
pointInPlaneY -= cY;
pointInPlaneZ -= cZ;

dot1 = (planeNormalX * pointInPlaneX) + (planeNormalY * pointInPlaneY) + (planeNormalZ * pointInPlaneZ);
dot2 = (planeNormalX * fX) + (planeNormalY * fY) + (planeNormalZ * fZ);

float t = dot1/dot2;

fX *= t;
fY *= t;
//we don't need the z coordinate in my case

return CGPointMake(fX + cX, fY + cY);


reacting said...

Can you post your conversion of gluUnProject?

Paul said...

Hi, I've converted the gluUnProject (I hope) and I'm trying to use your code to work out the actual coordinates in the 3D world, but the result I get back is 0,0. Any thoughts where I should check first, is it likely to be my conversion of UnProject?

Thanks for any help Paul

Ilias said...

I am having some problem getting valid gluUnProject point values as I am also implementing a glLookAt function. Anybody faced similar problems?

Paul said...

llias - my problems were firstly in my conversion and finally in the way I was saving the viewport data. My app allows the user to move the view around and herefore the viewport can change, I moved the storing of the viewport into the main draw code and all works ok

nonyabiz said...

man this conversion tough i am going use it for my custom term papers it is going to be tough explaining this thanks for this blog

Louis said...

For gluUnProject, have a look at (not that I've personally tested this yet, but there's a function in there!)

s. p. stanley said...

Question: Do you get the model view matrix after you perform your gluLookAt()?

rez said...

Hi Xenon!

It would be possible that you post a download link to the modified libGlu.a and libGLU_arm.a?

I tried to compile by myself but I really don't know how to do it.

Really thanks for your post.


rez said...

Hi Xenon!

Finally I managed to compile the modified GLU library and use your code to convert the screen coordinates to world coordinates.

Thanks for your post again!

daniel john said...

It’s great to see good information being shared.

Term Papers

Hiroshi said...

Maybe you can use a ray between camera position and only one point from glUnProject with zfar.

daniel john said...

There are certainly a lot of details like that to take into consideration. That is a great point to bring up. I offer the thoughts above as general inspiration but clearly there are questions like the one you bring up where the most important thing will be working in honest good faith. I don?t know if best practices have emerged around things like that, but I am sure that your job is clearly identified as a fair game.

Term papers

Ophir Paz said...

I used this sample to build a working sample for iPhone implementing gluUnproject :

Jerry said...

I think the interesting part of the project can be found in the glu.h / glu.c files and EAGLView.m.

College Term Paper

admin said...

iPhone is really the best.


Advantagepapers said...

The gluUnProject for iPhone / OpenGL ES is really one that looks so much good, I think that the given information about it is really good one to see and has so much to get know about.

College Term Papers

Custom Essays said...

I always like The I-phone, And gluUn Project for i Phone is really good one for the Users of I-phone. And The Information is really great. I am thinking to forward it to my brother.

College Term Papers

myterm said...

Thanks for doing such a thorough research work. I must say all the tips you have mentioned here are useful. So, I have saved the page. Hope it will help me in my future endeavors.

College Term Papers

sowmi said...

hi ,

i got a question regarding multiple objects drawn using push/ pop stack.something similar to this.






for (i=0 to 20) objects





each object is a unit cube (textured to sphere) with its own transformation.

In such a case how does ray picking work. How should i keep track of center point of the object to calculate ray intersection. i really appreciate any help.



Boundlesstech said...

Truly the blogging is spreading its wings rapidly.

Web Design | Web Hosting

hs said...

Incidentally, I like the way you have structured your site, it is super and very easy to follow. I have bookmarked you and will be back regularly. Thank you

Study Consultants | Study Agent

H S Consultants said...

Truly the blogging is spreading its wings rapidly.

UK Education Consultants

Boundless Technologies said...

If you want to see the mind blowing article with real facts and figures, this has really tremendous impacts on readers.

Web Design

Spiritual gifts said...

Thannk you for providing a thourough explanation for OpenGL. I am trying to fix a glitch that stops it from my iPhone.-spiritual gifts

Anonymous said...

I really agree with the facts that you have shared on this post. An interesting topic like this really enhances reader's mind to have more effective decisions over a certain issue.

CCTV Karachi

Term Papers said...

College term papers
No dispute with the text, agree with you. The features are really appreciable.

Boundless Technologies (Pvt) Ltd. said...

This is not the first of your posts I've read, and you never cease to amaze me. Thank you, and I look forward to reading more.

karachi online shopping

Boundless Technologies (Pvt) Ltd. said...

Valuable information and excellent design you got here! I would like to thank you for sharing your thoughts and time into the stuff you post!! Thumbs up

karachi online shopping

Jacob said...

Unproject twice to create a ray, ofcourse!!! Thanks =)

I creating a top down rpg, and want to click where the player is to move. So I create the ray, then find X and Y where Z = 0. Works great =)

Arsalan said...

This is a fantastic feature, other features are really fabulous though.

Custom written term papers

Sabra Allsbrook said...

Beautiful job! I just think they are brilliant. Thanks for this nice post.

cheap hotels in kansas city mo

Melany Flemmings said...

Interesting post. It is so awesome seeing young people love the world. Thanks for your thought and effort.

carry on luggage sizes

Crackshot69 said...

what would the code be to get the Z coordinate?

Crackshot69 said...

what would the code be to get the Z coordinate?

Marry Jones said...

I never read such beautiful article before. This article is very unique and full of surprise. I never read this kind of article that is straightforward when regarding to the topic. Very intense kind of writing. Keep up the good and unique article writing.

business plan service business plan writing service

Marry Jones said...

Author is working very hard to gain such a thinking. I'll use this when I'll work on. Nice to find one more talented person. This is an totally good material you have going here.

business plan writers | business plan writer

Vishal Tavande said...

Can I have the same code for android.???