Video games today are a far cry from where they began, but from Pong to the latest motion capture and 3-D graphics technology, engineers have been the backbone of this exciting and entertaining industry.

Early video games were entirely the creation of computer and software engineers. As the video game became more complex, their development required people from many different disciplines, such as graphic designers (who design a game’s graphics), game designers (who design the gameplay, rules, and logic of a game), composers (who score a game’s music), and writers (who write a game’s storyline and dialogue). Not only do engineers program the core of each game’s functions, technology developed by engineers determines how much freedom and creativity every other contributor can have in doing their job.

Engineers have played a huge role in the impressive and quickly growing technologies of video games. Today, video games can be played on many different devices, such as personal computers, mobile phones, tablets, and multiple video game consoles, each with their own unique operating system and approach. Video games also use several input devices, such as mice, keyboards, console-specific wired and wireless controllers, and motion- and heat-sensing cameras.

One of the main ways new video game technology is felt by the player is in the increasingly realistic and impressive graphics and sound that games offer. While graphic and sound designers deserve much of the credit for these, they wouldn’t have been possible without the help of engineers. Engineers created the tools and programs that allow designers to build advanced, 3-D, and motion-capture graphics. They also created the programs and systems that allow for higher quality and fully integrated sound effects and music.

Engineers are needed to program the core engine of the game—the basics of how it works, how objects move, and how graphics are displayed. But as engineers have expanded the technology available to video games, and as this technology has become more advanced, more (and more specialized) programmers have been needed.

3-D graphics programmers specialize in the creation of 3-D graphics. These engineers use complex mathematical concepts, like quaternions, linear algebra, and vector and matrix math to render these impressive graphics. Other programmers focus on sound. While almost all games, even the simplest of them, involve sound, some games today have their own musical scores, and employ advanced technology like 3-D positional sound, which produces sound using a variety of sound sources positioned throughout a three dimensional space.  Sound programmers often create and maintain the tools used by sound designers to assign different sounds to different events, characters, and actions, along with applying atmospheric music to different moments or settings within the game.

Physics engine programmers work specifically to develop the physics needed in a game. Different video games require different types of physics to be simulated. Many games will require that gravity, density, and water viscosity be rendered, while sports, racing, and fighting games can bring up more specific physics needs.

Certain games require an artificial intelligence programmer—someone to develop the game’s logic in order to develop the actions of the player’s computer-run opponents. Other programmers specialize in programming the user interface for a game or group of games.

As technology has advanced, many games and gaming systems now allow players to team up or compete using the Internet. This has created the need for network programmers who focus on developing the network systems of a game.

Other engineers specialize in creating the tools used in the building of the games themselves. These tools are used for things such as scripting, building game levels, and importing and converting art, and can make everyone’s jobs much easier.

Porting programmers specialize in converting a game from one platform to another, so that mobile phone game can also be played on your PC, or the hot new game is available for people with a PS3 and an Xbox. The porting programmer needs to know the two operating systems and their languages in order to convert all the different elements of the game successfully.

Video games are not only fun, they’re often at the cutting edge of new computer, interactive, graphics, and sound technology. Engineers have been, and will continue to be, major contributors to video games, improving technology and making even the most creative ideas into on-screen reality.

Simon Lui is an Assistant Professor of Information Systems Technology and Design (ISTD) at the Singapore University of   Technology and Design (SUTD). In 2008, he founded EC2 Hong Kong to develop iPhone and iPad  apps and sell them in the iPhone app store  worldwide.  Simon developed 7 apps, including #1 bestselling apps in several countries. Most of his apps are daily utilities and mini games that sell for US$0.99-$1.99.

Simon’s apps reflect some of his interests.   For example, Simon has a great interest in music, and minored in music as part of his undergraduate degree.   So, he developed “ec Shamisen” — a musical instrument app. so users can play the traditional Japanese instrument “Shamisen” on the iPhone. Also related to Simon’s interest in music and sounds is “SoundMitate,” a sound imitation game where players choose a sound and see how well they can imitate it. This app also lines up with some of his current research. Simon plans to develop this concept further into other useful apps such as a language learner.   And, he also developed Tinha War, which is one of his bestselling games.  The idea came from a traditional Hong Kong 1980s childhood paper game.

Q:  How  did  you  decide  to  study  computer  science?    Was  it  a  tough decision?

Screenshot from ec Shamisen. Image Credit: Simon Lui

In 2000, I choose computer as my major because I have many dreams to achieve. For example, to invent an “intelligent music search engine,” and to be one of the characters in a computer game, etc. I think the digital virtual world provide me the platform to do so.

It was not that tough to make the degree decision, since I believe the Chinese proverb. “Every profession produces its own leading authority.“ No matter which subject I take, if I enjoy it and work hard, I believe I can be successful.

Q: Did you play lots of electronic games as a child?   What was your experience then?

Yes I did play a lot. I got a Nintendo in 1988. I remember the game I played most is the “Fire Emblem” released in 1990. I am still playing their latest release on the Wii and NDS platform. This game impressed me a lot; it is a strategy-demanding game with strong story background. On the other hand, I also love the “Pac Man”, since I can just play it without thinking. I think both “simple” and “strategy- demanding” games have strong markets.

Q: How did you get interested in gaming as a career?

I hoped to work on something that I enjoyed. I loved playing games, so I wanted to develop my own creation. I need it, so I write it!

Q: Where do you get your game ideas from? What is your inspiration?

Screen shot from “Tinha War.” Image credit: Simon Lui

Many ideas come from daily life. I remember once I said to my girlfriend: “I will rescue you when you are in danger.” But she complained: “When?” So, I make it happen in a game! In “Tinha War” stage 24, the princess is trapped, the hero is going to save her. I am actually the hero and she is the little princess! Tinha is a Hong Kong traditional board game, which was very popular in 1980s-1990s. In this game, you will lead an army, aim to destroy the enemy bases. In the game you can draw your own solider, or capture an image from a photo.

Q:  Did  your  formal  educational  training  prepare  you  for  your  current gaming work? Why or why not?

Yes. I learned programming, algorithm, design pattern, and most importantly, I learned “how to learn.” In the world of computer gaming, we need to catch up with the latest technology all the time.

Q:   You started a company to help market your games…did starting and running your own business take away from your game development time? How do you balance the work?

Since I run a small-scale company, the business workload is not that heavy. Also, the good thing about selling products on the iPhone app store is that I can do business at any time online. For example, discuss with blogger via email, purchase Facebook advertisement on web, etc.

For game development, I keep the creative and high-level jobs for myself, such as game design and code architecture. I leave the time consuming jobs such as programming and data input to freelancers.

Actually, other than this app company, I also have another business on going, but I didn’t find much pressure in time management. Nowadays doing business online is really efficient.

Q: What challenges did you face when trying to market your games?

It is difficult to get my product in front of target customers. There is too much information on the web, so I have to work to attract the potential customers and stand out from my competitors online. In Hong Kong, we can make use of newsgroup; in the U.S., perhaps it is better to do it via technical blogs. In Japan, better to use a lovely girl as advertisement thumbnail….so marketing strategy can be different in different countries.

Q: Do you often have to redesign a game or app?   How many versions do you create before a new game is ready for launch?

Usually, I develop an alpha version for my friends to try it out. Then follow up with a beta version for a focus group to test — and then I will release the game.

Screenshot from SoundMitate. Image Credit: Simon Lui

Developers can update their app on the iPhone app store at any time. Usually it takes 7 days for Apple to review your update. Then the latest version will be automatically downloaded to all your customers’ iPhone. I revise my game frequently by considering customer’s feedback. One of my apps has had 17 minor updates after it  is  released,  and  my  customers  are  happy about that – they love to be listened to!

Q: You have many degrees….why did you decide to go for PhD?

It requires a Ph.D. degree in order to be a professor. Ph.D. is my final degree.

Q: How long have you been a member of IEEE?   What prompted you to join?

I have been an IEEE member for 5 years. I had my first academic publication in IEEE Transactions on Multimedia in 2006. IEEE membership is essential for my academic career.

Q: What is the most rewarding thing about the work you do?

Once I saw a stranger on train, he was playing my game happily with his friend. I am delighted to find my game did entertain people in real life!

Q: Can you share a story about how the work you do has impacted the world of gaming?

To be frank it is not me who has impacted the world of gaming, but rather all those developers who write apps on the mobile platform that did the job. For example, I was one of the first developers who released games on the iPad. Before that, no one carried such a big console and played game on the go. Thanks to iPad’s processing power and mobility, many people are now holding their tablet computer and playing games on trains and buses and in cafés. Also, with its big screen, more types of game start to appear. It enriches our lives.

Q:  What  advice  would  you  give  a  pre-university  student  who  was interested in working in the gaming industry?

The industry will keep on expanding, but the focus can be changed frequently. For example,  customers’  primary  interests  are  switching  from  household  game consoles, to desktop Internet games, and then to mobile consoles, and then to mobile games with Internet access…

For a pre-university student, make sure you are equipped with fundamental technical knowledge during school days, and “learn how to learn”: be able to search for answers on your own, and discover interesting topics and ideas. Then, you will be able to catch up with the ever-changing but exciting world of gaming!

Another IEEE technical society that gaming professionals might belong to is the IEEE Consumer Electronics Society (CE Society). The focus of this society was once limited to television, home hi-fi, and home appliances — but now it has expanded to reflect all the consumer products that integrate electronics! Members are involved with everything from gaming systems and equipment, to interactive information and display systems, home security systems, phones, music electronics, and much more.

In fact, the CE Society sponsors the IEEE International Games Innovation Conference each year, where all the latest in gaming is shared among developers — it’s a great chance to see what’s new, hear from gaming industry leaders, and be inspired by others working in gaming. Find out more about the 2012 conference online. (http://ice-gic.ieee-cesoc.org)

Step One: It’s best to start small and build a simple game like “pong” just to get used to the software. There is a nice tutorial that uses fruits instead of the original “pong” ball — that might be a good one to try.

Step Two: Think about something you enjoy doing and write down some ideas for a game that might be fun to do that relates to something you know a lot about or that you enjoy. Your characters can be anything you want them to be — and you can use clip art — you don’t have to be able to draw to make a really fun and nice looking game.

Step Three: Try using the software to create your own game. You might do a simple release to friends and then keep changing it or improving it over time. Some app designers listen to user suggestions for how they would like to see the game improved over time — this is a great way to get ideas for how users might like to expand your original idea. You can also do this activity with a friend or family member, taking turns adding features and testing.

Questions:

1. Did the tutorial help you get a feel for how the software works? Software engineers often build tutorials to help users learn how to use software.

2. How did your original idea of for your own game change once you tried to build it? Why?

3. How long do you think it would take to develop a new piece of word processing or graphic software? How many people do you think it might take to work on the engineering team to build this type of software? Why?

4. Did you find that it was easier or harder than you thought to program a computer game? Why?

5. What challenges did you face in building your game?

6. Do you think it would be easier or harder to develop your game as part of a team? What are the advantages of teamwork vs. working alone?

There are loads of games out there to play, but just a few that can give you a chance to see what engineers do. TryEngineering.org has developed several games that let you be the engineer — you can have fun playing and learn at the same time!

At http://tryengineering.org/play.php you’ll find a Bionic Arm Design Challenge so you can virtually build and test your own bionic arm design. There’s also a fun and fast paced trivia game called Questioneering, where you test your engineering knowledge and can play against friends or just go for the high score. Or, try out the Solar Car Racing game and design and race a car that will perform well on different types of tracks.

And that’s not all…there are dozens of games linked from TryEngineering.org that allow you to do everything from building a lifeboat, to designing a roller coaster, to crafting a trebuchet to destroy a castle wall. There’s also a fun game called “Invention Connection” through which you can trace the paths of innovation from the microwave to the mouse.

With so many games to choose from, it may be tough to pick one — but give a few a try and learn about engineering while you have fun!

Credit: Ergun Akleman

At last year’s conference Ph.D. student, Janet Marsden, presented her green ideas. Marsden foresees the day homes and businesses have their own solar, wind and geothermal power-producing technology, as well as batteries to store excess energy. She thinks the electric-vehicle battery technology being developed by automobile manufacturers will evolve into applications for buildings! “You make the house itself a charging station and you do the load balancing by putting the batteries there,” Marsden said. She added that because the U.S. electric grid is more than 100 years old, “we really need to look at what 21st century electrical generation needs to look like.” You can listen to more from Marsden here  and find out about the 2012 conference and also read about different green technologies at www.ieeegreentech.org.

Newman assembled a team from Lockheed Martin and the National Renewable Energy Lab in Golden, Colorado to retro‐fit an ambulance with rooftop solar panels and batteries. The goal of the solar system was to provide enough power to operate fundamental equipment while idling so that the engine could be turned off to save fuel. The flat ambulance roof offered 83 square feet of space on which they installed four solar panels which could generate 1000 watts. “The medical equipment only needed 463 watts to stay charged, so we actually over‐designed it,” Newman explains. “We found that each ambulance wastes about ten gallons of fuel each day just by idling. The solar panels can save that fuel.” At a cost of about $20,000 per vehicle to install the solar equipment, if each vehicle saves 10 gallons of fuel a day the solar panels would pay for themselves in less than two years.

Q: How did you get interested in applying solar power to ambulances?

Newman:  This project emerged as a result of conversations with ambulance drivers who were concerned about being ready to respond to emergencies around the city of Denver.  Currently, they are idling at about 40 locations around the city and burning expensive diesel fuel to keep the mobile hospital ready to deploy.  For me, this was a great experience since I was involved from the beginning to the deployment and was able to learn a great deal about the complexity of working with large groups of individuals.  It was great that we were able to have it operational in 6 months.

Q: What inspired you to integrate solar panels on the ambulances? Why was making them “green” so important?

Newman:  The inspiration for putting solar panels on the ambulance came after an analysis of options to make the fleet more efficient and less toxic to the environment.  Several ideas were discussed from making them into hybrid electric to using alternative energy.  Solar power is clean and available throughout the day.  The use of the vehicles at night requires storage of power so they can maintain operation.  Due to the number of hours they sit idle, it was important to have some way to reduce the amount of burned fuel.  The summer is bad since the vehicle must be in high idle mode to run the air conditioner so even more fuel is burned.

Q: What technical challenges did you face in incorporating the solar panels?  What had to change?

Newman:  Technically we had some issues wiring the panels to the charge controller and battery.  Some of the specs on the vehicle weren’t documented so we had to pull it apart in some areas to see where to run the cables and get to the battery.  There isn’t a lot of free space on an ambulance so the battery area required design of a shelf and tray.  Mounting of the charge controller was also tricky so we just put it on the door.  We also had to reprogram some of the interfaces to the battery so that it didn’t discharge when the vehicle was in the garage.  There were a few times it didn’t crank after being out of service for 12 hours.  After working with the paramedics for a few months of operation, we developed a set of steps for maintaining the system and this reduced a lot of the issue.  We also put in a GPS and power usage logger to see where the vehicle spent most of the time in order to fine tune the panel interface.  This gave us a good bit of data that could be used if we were to expand to the entire fleet.

Q: Can you explain a little about biomonitoring? 

Newman:  My research group works closely with physicians to identify problems that can be solved with the creation of embedded systems.  I am now part of the task force on with engineers and physicians based on a presentation at the 2^nd AMA-IEEE Medical Technology Conference.  Current emphasis is on technology at the point of care in the areas of limited mobility, limited sensation, rehabilitation, and chronic illness.  These projects take time to develop and deploy since we are bridging between traditional engineering disciplines and the medical profession.  We have systems deployed for fall monitoring at an assisted living community in Colorado Springs and are going through the process to evaluate a prototype at a Denver Hospital to aid individuals after polytrauma to communicate with care providers and loved ones.  My students really enjoy seeing the transition from the lab to the real-world and have helped me over the last 7 years pursue these topics.

Q: Do you find that “going green” has economic advantages too? 

Newman:   These vehicles are in operation for 12 hour shifts and just being able to save fuel is well worth it for the retrofit.  After doing a quick energy audit of the loads, estimating the typical idle patterns on the vehicle, and logging the mobility patterns of the vehicles over a period of a week, it was determined that a solar system could pay for itself within about 2 years.   Costs will continue to drop as the technology matures so it is definitely worth it to go green.

Q: How did you decide to go into engineering as a career path?  Was it a tough decision?

Newman:  Pursuing engineering was natural to me since I grew up taking things apart and wondering how they worked.  This sounds a bit cliché but I grew out of playing with dolls at a very early age and started working with my dad and grandfather on woodworking projects.  In middle school, my interest in mathematics, science and industrial arts expanded as well as my growing interest in music.  By the time I got into high school, I was weighing my choice between veterinary medicine and my passion for computers.  My dad purchased a Vic 20 for me and upgraded to a Commodore 64 for my entertainment.  While my other friends were just playing the Atari, I was learning to program in Basic and getting on the bulletin boards to download games.  It was fun and challenging at the same time.  During my third year in high school, I decided to pursue engineering which to me was something that I had never heard of before.  My high school had a computer club and also a robotics program so it was easy for me to get my hands on interesting systems, learn about logic, and do more detailed programming.  We were an engineering magnet school so I had the opportunity to meet with people working at industries around my hometown and I also went to science fair competitions.  The project I created for my final year ended up at the national finals and there I got to meet even more researchers and discuss my ideas and plans for the future.  It was pretty clear to me that I wanted to be an electrical and computer engineer.

Q: You have many degrees….why did you decide to go for PhD?

Newman:   When I was a junior in college, I took a class from a Professor who was doing a lot of research in my areas of interest.  He liked my work ethic and persistence so I was given the opportunity to work in his lab.  Based on this experience, I studied in France at the start of my graduate education and continued research with other faculty to expand my knowledge.  In the meantime, several large research centers emerged on campus in Atlanta so I was brought back to continue to develop my topic as a graduate student.  Our work was tightly integrated with industry needs and there was a good flow of interaction between project work and classroom experiences so I didn’t hesitate when an opportunity to pursue the PhD was provided.

Q: How long have you been a member of IEEE?  What prompted you to join?

Newman:  I think I joined as an official member in 1999 when I finished my Ph.D.  Before that, I participated in events on campus as a student and started to read the magazines.  When I got my first “real” paycheck I was happy to invest in an organization that is beneficial to humanity.  It has been very beneficial to me over the years and I strongly recommend it to my students to join as soon as they can.  One really nice part of the organization is that it isn’t like a lot of the fraternal societies.  Women and men are equally recognized and promoted based on their technical contributions.  This was very appealing to me.  The contacts that I have made at conferences, along with my participation in the societies, has been very rewarding.   I was able to participate in the Computer Society International Design Competition after meeting the organizer at a “Frontiers in Education” conference.  Students came up with some really creative solutions using the Bluetooth development kits and software platforms from places around the world where this technology is very hard to obtain.  Reading their reports and interacting with my colleagues to shape the competition was very helpful for me to have a fresh perspective on engineering around the world.

Q: What is the most rewarding thing about the work you do? 

Newman:   I really enjoy helping people.  This includes my colleagues, students, and members of the public.  If I can tackle a complex problem and break it into pieces that are easy to explain, I think my goals for the day are met.  There are so many new and emerging computing platforms that can be harnessed for the benefit of humanity.   It just takes time and patience to understand the need and match up with the capabilities.  We would joke about engineering tomorrow the night before but sometimes that is the case.  You have to be ready to respond and have the team with you to do so in a safe and effective manner.

Q: Can you share a story about how the work you do has impacted the world?

Newman:  I am a little modest about my own accomplishments but I think the success of my students is my major contribution to the world.  The “ah-hah” moment is something that is hard to describe but keeps me energized for the next project and challenge on the horizon.

One of my students went with me to an international biomedical conference.  He presented a poster on a system he created for pressure sore monitoring in a vehicle for individuals with diabetes who had peripheral neuropathy.  The attendees were amazed with the clarity and complexity of his presentation as well as the questions asked during the sessions.  He is currently considering a PhD.

A few years later, my lab was invited to develop a book chapter for ambient assisted living with international collaborators as a result of connections made at this conference.  We have since generated IP that is going through patent protection and implemented a test system to help individuals at risk of falls to show the proof of concept.  I am also now working with international collaborators on additional research projects in the biomedical area as a result of this initial project with my former student.

Q: What advice would you give a student who was interested in working in a “green” field that involved engineering or computing?

Newman:  Go for it!  The need for sustainable systems is on the rise and will continue to grow.  Engineers with skills in electrical, computer, and energy have an advantage and will find many opportunities to pursue exciting career paths.  The power of computing platforms provides many opportunities in the last 20 years.  As we move into electric vehicle, high penetration solar and wind, as well as smarter use of technology for medical applications, the integration of these systems into everyday use will continue to provide opportunities for young engineers to make their mark on society.

A solar car is a vehicle which is fueled by the sun’s energy. A solar car has photovoltaic cells in its solar panels’ surface that converts the sun’s energy into electricity. This electricity either runs a battery that powers the car’s motor or powers the motor directly. Today, most solar cars are used for racing — so why not try your hand at a virtual solar car race? In the “Solar Car Racing Game” you’ll select important components of your solar car that can impact its performance such as the battery and tires.  You’ll try out your design in both a desert and a racetrack environment and see how it works.  You may choose to change your design to improve its performance and then race it again.

If you find you enjoy designing and testing solar vehicles in a virtual environment, you might also want to join in on some real competitions where you can work with other students to create a real solar vehicle.  Some suggested competitions include the Junior Solar Sprint Competition, which is a design engineering challenge for students in fifth through eighth grade design and create model solar electric race cars. At the university level, the World Solar Challenge draws teams from all over the world to Australia, where they test their solar car designs in challenging environments.  And, the Hunt-Winston School Solar Car Challenge is a closed-track event for high school teams that is held at a motor speedway.

What will the houses of the future look like? If the Earth gets a say in it, they’ll look a lot like what engineers are working on today—sustainably built, sometimes self-reliant ecosystems that make the smallest environmental impact possible. While breakthroughs in green technology can impact the environment in all areas of life, looking at how we design, build, and run our homes and other buildings can make a major impact on the health of the environment. It has been estimated that current buildings are responsible for 40% of the world’s total energy consumption and for 24% of global carbon dioxide emissions.

Engineers around the world have a hand in the future of the environment through the development of green technology, greener and more sustainable materials, and new methods of energy production. With greener technology being developed constantly, the house of the future is looking much more Earth friendly.

The process of green building begins with clever design. Calculating something as simple as where to place the windows can make a major difference in energy usage. Strategically placed windows can mean fewer hours using electric lighting and less need for heating in the winter. Shading of windows and roofs in the summer can mean less energy use and less pollution from air conditioning. The same can be true of the placement of insulation in walls, ceilings, and floors.

Technology that makes small adjustments to a homeowner’s energy consumption can make a major difference as well. Connecting lights to photo sensors, so they only come on when someone is in the room, can help to conserve energy. Thermostats that can be programmed and can work on timers can mean heat and air conditioning aren’t being used when no one’s home or when they simply aren’t necessary. Similarly, water heaters that can be programmed and work on timers can allow water to only be heated when it will be used, rather than heating it constantly, even when nobody’s home.

Green energy production technology built by engineers mean that homes and other buildings can also be designed to produce their own energy through the use of alternative energy systems like wind turbines and solar panels (photovoltaic systems). These devices are able to convert natural energy sources, like wind or sunlight, into electricity that can be funneled straight to the building’s electrical grid. In fact, some buildings are designed to be so energy efficiency that they actually produce more energy than they use. These buildings don’t need to rely on power plants, which saves the owner money on electrical bills, doesn’t use limited and polluting fossil fuels, and keeps carbon dioxide out of the atmosphere.

Materials engineers have a huge role to play in green building. Developing building materials that are recycled or sustainable, durable, and still cost-effective and appealing to homeowners, mean more people will choose to use them. Engineers are also able to think about how a material will be reused or disposed of, when it’s no longer needed, and can work toward developing materials that won’t leave a negative environmental impact after its use. Discovering ways to build more energy efficient household elements can make a major difference in energy efficiency and pollution. More efficient insulation, insulating windows, and energy-efficient appliances can all make a big positive impact on the environment.

Computer engineers and software developers have a role to play in green building as well. Home energy consumption monitors can make homeowners more aware of their energy and water usage and more likely to conserve these resources. Software has also been designed to help builders find the right green materials for their projects. Software like the ATHENA ® EcoCalculator for Assemblies and BEES ® Building for Environmental and Economic Sustainability allow builders to test out different combinations of materials and look at both environmental and economic factors to help them decide what materials will be right for them.

The environmental concepts that go into green building can be applied to all facets of life and engineering. Developing new technology that assists people and businesses in making more environmentally responsible choices, engineering new materials that will have less of an environmental impact and can help conserve energy, and thinking of unique ways to change our current systems of living and working to make them greener will all have a major impact on the health of our environment.