Check out our new website: WiiR3D.
Saturday, August 2, 2008
Sunday, May 4, 2008
Elaboration Excitement
News
It's all action in WiiR3D's elaboration phase. We have been working towards creating an architectural prototype and extending on Johnny Lee's target room to build a prototype of our Menu room.
As you can see, we've had a go at building some basic 3D models and the perspective control still needs some work, but we're definitely making progress.
We have also received, from the very kind people at OpenKMQ, a new headset and Wiimote holder. It is very lightweight (PET plastic), designed with head tracking applications in mind, and definitely an excellent option for anyone building their own IR LED headset. We are particularly fond of their Wiimote holder, which mounts easily on a flatscreen monitor. This position for the Wiimote is ideal for when users are playing a game while sitting at the computer.
We still are looking into options for a way to mount the Wiimote when the user is playing is a standing position - a kind of retort stand has been suggested, as well as something based on a camera tripod.
In other news, our official website is now under construction, as you may have noticed. You can visit via the navigation bar at the top of our blog, or access it directly. We will be working to have the site fully functional within the next few weeks.
From the WiiR3D team
Jessica Lloyd
Hello all! My work lately has been quite engaging - I've enjoyed trying my hand at 3D modelling as well as webpage design.
No-one in the team has worked in 3D modelling before, so giving that a try was quite a learning experience. I'm still in my early stages, but I did build the objects that you saw in the Menu room, and have been experimenting with some more organic-looking objects.
It's also been my pleasure to put together our official website. Google's been such a good host for our project thus far, it seemed natural to utilise GooglePages. I haven't designed a web page before, but Google's simple WYSIWYG editor helped me build (what I'd like to believe is) a professional site to promote the WiiR3D product.
Adam Parkes
Hi Everyone,
Things that I’ve been doing over the last month include:
To build the initial 3D models I used Sketchup. Sketchup is a program that makes developing 3D models extremely easy. There are some problems with Sketchup: it's buggy and it can’t directly save to FBX format which is what we required. There was a simple way around this which involved exporting the model to a format, renaming the file as a zip and running a converter package over a particular file in the zip. Aside for the above annoyances I did really like Sketchup and I would recommend it to anyone that knows nothing about modeling.
My goal was to place the models I created in the environment, this involved learning about the 3D pipeline. I followed tutorials such as http://msdn.microsoft.com/en-us/library/bb197293.aspx. The process was not too tricky, although it was time-consuming. I made a framework for the models called an entity that reduces most of the repetitive actions that have to be preformed on models to get them to display.
I helped develop the initial structure of the server. This involved deciding on communication protocols, system structure and the division of tasks between the server’s components. I was also involved in the initial testing of the server generating data follow and displaying pseudo points.
This interface was one that I developed and will continue to expand. This interface will assist any developer that wishes to use head tracking hardware other than the Wiimote. To use other hardware developers will have to implement this interface on their driver class. The server has a constructor that takes an instance of a class that implements the IHeadTrackingDriver.
I developed the WiiHeadTrackingDriver. This class is responsible for connecting to the Wiimote and reciving data for the IR coordinates.
I developed this class as a method to test the connection of the server and client. I wanted to be able to test if we could pass coordinates through to the client and at what speeds. This class will be used over and over when testing the server. In fact I will extend this class in the future to create an ErroringHeadTrackingSimulatorDriver. The Erroring simulator will be very helpful when testing the robustness of the server.
Currently I’m working on intergrating the wiimotelib into our program so that we can use it as a secondary input device for the games. The biggest problem that I’m currently having with that is that the nunchuk will not connect to the wiimote automatically.
In the future I will:
James Leskovar
Progress report [5/4 – 17/5]:
Tasks I was responsible for over the past month-and-a-half include:
• Spiking XNA
The goal for this task was to play around with C# and the XNA framework, in order to determine its feasibility for the games we’ll be building, as well as to establish a baseline architecture if it does prove feasible. A basic "cube grid room," was constructed, which utilizes the same perspective projection manipulation performed in Johnny Lee’s DesktopVR application. Issues regarding the correctness of the perspective projection arose during development (see above.) Additionally, it was decided that whilst XNA does provide a suitable abstraction over the graphics device and 3D rendering, it may be more appropriate to use an engine which would allows us to quickly develop, test and modify some sort of 3D environment. To this end, it was decided that the Panda3D game engine should be used. Panda3D is written in C++, though can be fully utilized from within the Python scripting language.
• Head-tracking gear construction
Following the advice of Christian Muise, the 9V batteries from the prototype head-gear were replaced with 3V (2025) watch batteries, in order to achieve a smaller form factor. Currently, the head-tracking gear consists of three small infrared LED circuits attached to a pair of safety glasses. Additionally, a shipment of TSAL 6200, 6400 and 7400 infrared LEDs have just arrived (thanks Jess!), which I intend to experiment with. The TSAL line of infrared LEDs are recommended by Johnny Lee, and so it would be interesting to see what differences (viewing angle and range) they make when compared with the stock Dick Smith Electronics infrared LEDs.
Also worth mentioning is the equipment received from Pixel Partner, in particular, the reflector-based glasses and the Wiimote monitor stand. We found that the reflector-based glasses were too flimsy to wear comfortably, due to the type of plastic (PET-G) used in its construction. However, we found that the Wiimote monitor stand proved to be an ideal solution for the placement of the Wiimote.
• Head-pose tracking algorithms
Potential head-pose tracking algorithms aimed for three-point tracking were investigated. Two worth mentioning are “Simple, Robust and Accurate Head-Pose Tracking Using a Single Camera” and POSIT. The former paper describes a method of pose-tracking via root approximation of parametric equations representing the points in 3D space. The latter is used in the FreeTrack project. Fellow WiiR3D member Buu is working on the head-tracking computation, see below for more information.
• Head-Tracking Server
The WiiR3D system is comprised of a server-client type architecture. The server, written in C#, does the majority of the grunt-work, and is responsible for performing the gesture recognition, head-pose and distance calculations and broadcasting this over a datagram (UDP) socket. This allows developers to easily utilise our information from any language (compiled or otherwise), as long as they have support for sockets. Currently we have a functioning iterative server written in C# which is able to communicate with a listening client over UDP. For simplicity, a fixed-length protocol has been designed for all communications between server and client.
• Standard documentation (protocol description and basic operation for server, construction of the head-gear, head-tracking computation, configuration and usage)
Tasks to do in the immediate future:
• Spike the Panda3D engine.
• Investigate issues with perspective manipulation.
• Look into head gesture recognition
• Implement link-status in the server (rudimentary syn/ack over UDP)
• Multithreaded server.
• Improve head-tracking gear (parallel circuit, use TSAL LEDs)
VinhBuu To
Hello all,
Yeah, I don't have much to talk about since the other guys have talked about all the stuff we've done. During the elaboration phase, I was working with the mathematics for head tracking calculation and am still working on that. I also have enjoyed modelling for project.
My goal in the immediate future is to finish all the maths work and translate it into code so that we can use it for a real head tracking application.
Monday, April 7, 2008
Welcome to WiiR3D
Hello and welcome to the WiiR3D blog!
We are a third year project team from the University of Wollongong, Australia. Throughout 2008 we will be developing WiiR3D, a twofold product dealing with the implications of head tracking on the 3D gaming industry. We will be building on the research of Johnny Lee from Carnegie Mellon University in order to develop an API for developers to incorporate head tracking as a new input for computer games.
What is 'head tracking'?
Head tracking is the process whereby a pair of IR sources, typically mounted on the head or a similar position, are accurately located by an IR camera, such as the Nintendo Wiimote, so as to provide view-dependant images on screen. This means that the computer can know where the user is situated relative to the computer, and adjust its display accordingly.
What does head tracking imply to 3D gaming? That is the question that WiiR3D aims to explore. As you can see in the video, head tracking allows the computer to generate a much more authentic three-dimensional environment than is currently standard. This step closer to 'virtual reality' for the common gamer is a significant one, and even on its own, this new technology could revolutionize the 3D gaming industry.
However, enhanced realism in the gaming environment is not the only way in which games developers can utilise head tracking technology. Head tracking can also be used as an active control, and this is another function of head tracking that WiiR3D intends to explore. To do this, WiiR3D plans to develop a set of three minigames in which a variety of possible uses for head tracking will be demonstrated.
From the WiiR3D team
Jessica Lloyd - Project leader, resident scheduling phreak and (technical) writer
G'day, I'm Jess and I'm still suffering slightly from shock that I have finally made it to this stage at UOW - mixing computer science and creative writing degrees is rare, and it feels like I took an extra long time to get to here. Fortunately, I am lucky enough to be in what I believe to be the best team I could be, working on the most exciting project in which I've had to opportunity to be involved. I'm confident that WiiR3D is going to be a great success, and a fantastic learning experience for all of us - and I think that we'll be developing a product on which other developers will be able to build upon in the future.
I hope that you will find this blog informative and interesting. :) Adam Parkes - Testing manager and coding enthusiast
Hi all,
My name is Adam Parkes; I'm a third year computer science student. I love coding and gaming. I want this project to be successful and be something that I look back on in fondness. I also hope that in some small way we will help change and influence those of you who produce games and are reading this. I also hope that you follow us through this journey and learn as much as we learn as we try to develop an API for head tracking and a series of three demonstrative games.
parksey James Leskovar - Development manager and programmer
Hi there. I'm James Leskovar, and am a member of the WiiR3D team as part of the third year project at the University of Wollongong. I'm primarily interested in programming, AI, and computer architecture, so the project is definantly an exciting challenge for me. We hope the WiiR3D project will give hobbyist and professional developers alike an opportunity to enjoy the exciting new technology of head tracking.
VinhBuu To - Database guru and modeller extraordinaireG'day,
My name is VinhBuu To. I am taking part in the WiiR3D project as part of my final year at University of Wollongong. Personally, I am interested in programming and software modeling as well as databases. WiiR3D is really a big challenge for me, but I think we have the best team here. So, I hope what we develop in WiiR3D will be helpful and interest people.
Cheers,
VinhBuu ToLatest news
Inception
During Inception the WiiR3D team worked on understanding the scope of the project. James looked into the technologies that we will be using throughout the rest of the year, Adam got the requirements down-pat, Buu modelled plans for the overall application architecture and use cases for the game component, and Jess dealt with administrivia and meetings. James was the first to get Johnny Lee's sample head tracking application up and going, and built the following headset prototype:
As you can see, while the headset does what we need it to, it is slightly lacking in the area of style and class. Fortunately, having a stylish headset is not the most important part of WiiR3D, it's what we allow the user to do while wearing it.
Project scope
WiiR3D will be a software system which explores and demonstrates the implications of head tracking to the 3D gaming industry. Aimed at 3D game developers, our three rudimentary minigames will each demonstrate a different way in which head tracking can enhance the 3D gaming experience.
WiiR3D will use a headset mounted with at least two clusters of infrared LEDs to indicate the user’s head position, and utilise a stationary Wiimote to read the infrared information and transmit it via Bluetooth to the computer system. WiiR3D will then interpret this information in order to adjust the display on screen relative to the user. The system will also be able to deal with a secondary input device, such as a keyboard or a second Wiimote wielded by the user, in addition to head tracking.
Game component features
Main menu
The WiiR3D menu will demonstrate how the user can make simple commands using head tracking. The user will be able to employ familiar movements to command the application. These movements are nodding, shaking the head and nudging left and right, which roughly conform to the functions of the enter key, escape key, and left and right arrow keys respectively. The system will identify nodding, shaking and nudging movements using the infrared headset and the Wiimote’s infrared camera, and control the view and actions within the menu’s 3D environment according to the user’s movements.
We also have a quick mockup of what the hope the main menu will resemble:
Hide and seek
Hide and seek will demonstrate how head tracking can be used to allow a user to explore a three dimensional environment. The environment that the user will be exploring will see objects obscuring the ‘treasure’. Once the user has found the treasure the game will give some trivial feedback about time it took to find it. Head tracking will allow the user to view the environment in a manor that feels natural. The user will be able to move around in front of their monitor and see different perspectives of the virtual environment.
Pong
Pong is a traditional video game based on table tennis. The game environment is a 3D room where the ball bounces into the wall and the user has to catch the ball with their ‘paddle’ by moving their head. In this game, the user will move their head to move the ‘near plane’ in order to hit the ball. When the user comes near the Wiimote, he/she can have bigger ‘viewing frustum’ of the 3D room and vice versa.
Space Invaders
WiiR3D Space Invaders is yet another reincarnation of the traditional game. It aims to retain the retro feel of the original whilst adding many new features to enhance the game. One such enhancement is the use of full 3D graphics for game play. Additionally, head tracking will be used as a form of input, enabling the user to rotate their aiming direction.
Game play follows the original arcade version of Space Invaders. The user is presented with a first person view of the game field. Enemy critters surround the user from in front and above, and gravitate toward the ground at an increasing rate. The user’s goal is to prevent any enemy critters from touching the ground. The player is armed with but a single-shot laser cannon, and has a number of lives they’re able to lose to the critters before the game is over.
The user is able to rotate their view with 3 degrees of freedom using head tracking. This enables the user to look around their environment and precisely aim at individual critters, with the laser blast following directly down the user’s gaze. As with the other games in the WiiR3D package, head tracking will also be used as a form of menu navigation.
Lifestyle Objective Milestone Review complete - Welcome to Elaboration!
During this phase of project development we will be building an architectural prototype. Elaboration will stretch from now until 21 May.
Subscribe to:
Posts (Atom)