Accelerometrics:The art of motion sensing. Because motion is life.

 

We look around, everything is in motion. When motion slows, life pauses. Motion shapes our lives and gravity defines motion. We want to use gestures to control mobile devices: ShakeShake to roll virtual dice in the iPhone, TapTap to adjust the volume on the headset, Tilt ‘n Roll to navigate Google maps. We use motion to control our favorite devices and interpret motion to improve our sports performance and our health.

Gravity, Galileo, Newton and Us

Gravity keeps us grounded. It’s our blessing and our curse. Gravity is the key force that shaped evolution. Accelerometers sense gravity, giving us the signals to understand motion. The ancient Greeks with all their brilliance didn’t understand motion, gravity or time very well. More than fifteen hundred years later, Galileo became the father of modern science. Before Galileo, with the prevalence of “Greek Cosmology,” heavier things fell faster and the earth was at the center of the universe. Galileo changed everything. He was skeptical of all the grand Greek schemes. He simply used his pulse to measure time and rolled balls of different weights on the same inclines. He carefully marked where each ball came to rest with every heartbeat (about one per second). Careful observation showed that no matter what the weight was, the balls all moved following the same patterns: one unit in one heartbeat, four in two, nine in three and sixteen in four. Galileo created a repeatable and reliable experiment. Later he used water-clocks and pendulums to measure time more accurately. The Greeks were toast and the modern scientific method was invented: observe, reason and experiment. It was the dawn of the 17th century, some three hundred years ago. Newton then took all the pieces, trusted in Galileo’s principle of inertia and postulated that there had to be a force that attracted the Earth to the Sun and the Sun to the Earth and everything to everything else in the universe. We still don’t understand the nature of that force. But we know that gravity is everywhere, keeping the universe in balance.

Walking: Defying Gravity

Walking is so natural to humans that we forget the millions of years of evolution that allowed us to defy gravity by standing up and running on our hind legs consistently. Something that most children learn to do before they are two years old essentially defines us as a species. Our head, brain, intellect above all. Standing tall, making weapons, hunting in groups, using tools, and inventing technology has given us humans complete control of the planet. For the better and the worse. Consider this factoid: for every wild ‘protected endangered gorgeous wild wolf,’ there are more than 1000 pet dogs. The domesticated Canis Lupis seems to have prospered much better than the wolf in the wild. Smart move. We now get to use technology to try to better understand motion. After defying gravity, we get to measure gravity. And for this we use motion sensors. With smaller, more accurate and more power efficient sensors we can embed motion sensing in almost every mobile device. Now we are measuring and interpreting gravity and taking action.

Measuring Gravity with Sensors and Calibrating Human Motion

Thanks to Newton and Galileo, we know that we can understand motion by measuring acceleration. Cinematics, the science that studies motion, uses high-school calculus to relate acceleration, speed and position. In a mobile device, when we do this in real time we don’t really integrate acceleration to get speed and speed to get position. We interpret the signals of the sensors and recognize motion patterns. Just like a good speech recognition engine recognizes words captured by a microphone. We use innovative technology to interpret signals captured by nano-technology based sensors so that our mobiles tell us how fast we are running or walking for example. The science of accelerometers is profound and essential. Satellite based navigation systems are of little use when signals vanish in an urban canyon or a wooded area or when changes in position and motion are unrelated as on a treadmill. Accelerometrics is a cool new discipline. Newton and Galileo would love it.

Putting Life and Motion in Control

The Wii changed gaming consoles by moving them from the hardcore gamer community to the mainstream. My 11 year old daughter and I love to play Mario Kart with the rest of the family. Great stuff on a big screen. On mobiles, in the palm of our hands, it’s a different experience. TapTap, ShakeShake, Tilt ‘n Roll are naturals. Motion now controls a whole virtual mobile world. The motion-aware mobile platform is the new media. It is going to be by far the most popular platform in the world as it is just a matter of time before everyone of our billions of motion-enabled co-humans wants one. Because life is motion.

 

Will my favorite Android soon dream of sheep?

People use AI as a buzz-word to promote the Roomba vacuum cleaner: That’s a great sign! To be fair, it does fit into a definition of AI of a system that perceives the environment and can make intelligent decisions. The kind of decisions that a reasonable human would make. Now imagine if every camera-phone had the ability to “sense the environment” and make “intelligent decisions” that can anticipate and act meaningfully. Then AI comes as a way to help and enhance the lives of real intelligent beings: All of us!

AI is present in a system that perceives its environment and takes actions which maximize its chances of success. For example the next generation of sensor-enhanced mobile devices may use enough smarts to qualify as AI-based systems. At least that is what we are working very hard on doing at Fullpower.

AI is not just about systems that can learn. I think that for AI, what is more important is understanding the environment and making inferences that maximize chances of success. Learning can be part of the process. It is not necessary or sufficient. By the same token, natural language processing is not automatically AI. It can be. We can use AI techniques as part of a system that does natural language processing. But language is not automatically intelligence. It is communications.

For years, the Turing Test was seen as the criterium and the end of it all: If a human communicating using text messages with a machine wasn’t able to recognize that he/she was dialoguing with a machine, then that machine had to be “artificially intelligent”. The Turing test in my opinion is simply about building a machine good enough to be able to fool a human into believing that it is human through any text message interaction. It’s of course always an interesting exercise, but at the end of the day it does not attempt to truly emulate the advanced problem solving abilities of human intelligence. Let alone any form of “social-intelligence” or understanding of the environment via sensors for example. And conversely we can think of many humans who could fail the test themselves yet have “natural intelligence”. So the Turning test may just be an interesting exercise, but not a way to characterize machine or human intelligence.

As it is many times the case, I think that academia may have gotten a bit stuck with the LISP machines industry and with robots. The thought was to replace human intelligence and/or labor. However things are changing quickly and technology is moving by leaps and bounds. For example, when we all thought that robotics would allow American and European manufacturing to be more competitive, China has become the “factory of the world” without technology by leveraging an endless low-wage hard-working low-skilled workforce. Present day industrial robots are made of just a little bit of AI and a lot of electronics and mechanics. I’d take R2D2 any day! The world of sensor-enabled and enhanced devices with integrated inference engines has the greatest practical promises for AI’s long term success. Next generation robots get better!

Yes, I’d predict that most of the successful and useful advances will come from sensor-enabled devices and networks of such sensor enabled devices. Both will be important and make significant advances using sensor-enhanced solutions.

Yet, as Shakespeare eloquently says: “We are such stuff as dreams are made on.” Our robots and machines don’t dream yet. Or as Philip K. Dick’s masterpiece: “Do Androids Dream of Electric Sheep?” asks is the true test “emotional intelligence”?

Cover of first edition (hardcover)

Cell Phone, The Ring Heard Around The World

http://www.cbc.ca/doczone/cellphones/video.html

This piece is 41 minutes on the camera phone. There is even some stand-up paddle-surf and great footage from Asia.

The creation of the camera phone starts at about 21:10 and runs through 27:00.

Some highlights:

The story of the “Camera Phone Birth” at 21 minutes
Sophie from behind a camera-phone taking a photo at 23:06 (the story turned around!)
Footage from Fullpower’s office including Arthur and Philippe at 38 minutes with lots of details: Games, Sports, Fitness, User experience

 

Sensor-Based Phones are the Next Big Thing!

People are asking what’s next in wireless technology?

It’s sensor-based camera-phones that integrate motion, light and touch technology and will do for camera-phones what the Wii has done for game consoles. Opportunities like this only come around every 5-7 years and it’s here now.

For the last 5 years, Fullpower has been building breakthrough sensor-based technology with rapid deployment strategies, end-to-end solutions and a strong IP portfolio. I have to say that our team is very proud of our foresight in “seeing” the future.

We are inventing the future of mobile devices. That’s really exciting!

 

The future of camera-phones, the next couple of years.

Q and A with Philippe Kahn, CEO of Fullpower,
Creator of the first camera-phone solution

What’s next for user-handset interaction?

Philippe Kahn>> Motion is coming next . Shake your camera-phone to pick-up a call. No buttons, not fingers. Simple natural gestures. Tap it to advance songs in the media player.

What can we learn from devices like the iPhone, Nintendo Wii, Logitech MX Air that utilize accelerometers and 3D space sensors?
Philippe Kahn>> The Wii did a lot of things right. What you are going to see is a lot of what people hadn’t thought about for motion. Very cool, interactive and a surprise for many. It takes complex software technology to make it work right. All the prior attempts have been botched. Just like there were lots of touch-screen phones before the iPhone. But nobody had it right.
Will there be a change in how we use our handsets?
Philippe Kahn>> Camera-phones are communications tools. So that’s what we’ll continue doing with them: Talk, share, watch, play. How we do it gets more intuitive, direct and fun. Motion is a natural way.
Do users have a need for accelerometers and orientation sensors in their phones?
Philippe Kahn>> Sensors bring magic to many devices. But the sensors themselves can’t do anything without the breakthrough solutions. Accelerometers have deployed airbags in cars for 20 year, they just became useful for gaming. For the last 4 years dozens of team members at Fullpower have been inventing new ways to use sensors and bring a paradigm shift to camera-phones, MP3 players and many other devices. People have tried sensors in camera-phones before, but the software didn’t work properly and it all remained a science project.
What is FullPower working on to address this?
Philippe Kahn>> Fullpower has built a huge technology and IP portfolio and filed dozens of patents during the last 4 years. Fullpower works with market leaders to take their camera-phones to the next level of usability. The nature of the Fullpower business is that we can’t discuss the projects that we work on for competitive reasons. We are very discrete. In fact if you asked me whether our technology is embedded into one or another handset contractually I’d have to say: “No Comment”. 🙂
Do you think the cell phone will eventually evolve into a phone, internet device, GPS navigation system and more? If so, what tweaks will we need to implement to the interface?
Philippe Kahn>> There will be all sorts of camera-phones. Small, medium, large. All users are different. We will see everything in terms of functionality, form-factors and interaction models. Sensors embedded in all these camera-phones working with Fullpower’s software will make camera-phones smarter and much more aware of the environment they are in. That means a much cooler user experience. When everyone does voice, email, browsing, chatting, pictures, video…. What counts is user experience. And our vision at Fullpower is to radically simplify the user experience.

Baby’s arrival inspires birth of cellphone camera — and societal evolution

A decade ago — 10 lousy years ago — the cellphone camera was invented.

This is almost impossible to comprehend. The cellphone camera is now almost as much a part of daily life as toothpaste. On an increasingly regular basis, the technology alters world events, as when that Iraqi guard used his cellphone cam to record the hanging of Saddam Hussein. Imagine if cell-cams were around when Marie Antoinette went to the guillotine.

Motorola CEO Ed Zander told me that his company, which makes cellphone cams, sells more cameras than any camera maker. Gartner Group says that 460 million cellphones with cameras were sold in 2006. By 2010, the number sold per year will pass 1 billion. These things are moving the way McDonald’s moves hamburgers.

Star Trek always gets kudos for getting the future right, but those beam-me-up-Scotty communicators were pitifully camera-less.

The whole cellphone cam movement started, oddly enough, with one of the great, colorful characters from the flowering of personal computing in the 1980s: Philippe Kahn.

Back in those days, he was a large, contentious, French-accented jazz flautist who started software company Borland. After turning Borland into one of the major early successes in PCs, Kahn was pushed out in 1995 in a dispute about the company’s direction. He then launched cellphone software company Starfish, which played a role in his invention of the cellphone camera in 1997.

A number of companies were messing around with the idea. Putting a camera in a cellphone was becoming nearly as obvious as realizing butter should go on toast. But to make the concept work, somebody had to come up with the knife, so to speak. Kahn gets credit for doing that for the cell-cam.

Kahn’s story of the origin of the cell-cam is kind of cute. It started when his wife, Sonia Lee, roared at him while spending 18 hours in labor. “I’d gone to the Lamaze classes,” Kahn, now 54, tells me. “And the second time I said, ‘Breathe!’ Sonia said, ‘Shut up!’ So I said, ‘OK, I’ll sit at this desk and find something to do.’ ”

He had come to the hospital outfitted, as usual, with his laptop, cellphone and digital camera. He thought about how clumsy it was to have to take a digital photo, download it to his laptop, post it to a website, then e-mail his friends to tell them where to look — all of which was pretty new at the time. He wanted to snap a picture, hit a button and have it automatically load to the Web.

As his wife’s labor went on, Kahn started fiddling with his hardware and writing code to glue it together. “I had time to make a couple trips to RadioShack to get soldering wire,” Kahn says. “I just stayed in the room and made that thing work.”

By the time he was holding his newborn daughter, Kahn could use his jury-rigged contraption to take a digital photo and wirelessly post it for his friends and family.

Motorola was in the process of buying Starfish, and Kahn says he first showed his invention to his new boss. But Motorola was just getting a new CEO (Chris Galvin) and embarking on one of the most ill-fated projects in global corporate history (the Iridium satellite phone system). Motorola passed on the cellphone camera.

“Motorola was in turmoil at the time,” Kahn explains.

Kahn formed a new company, LightSurf, to build and market PictureMail — a back-end system that would let a cellphone take a photo and send it somewhere. The first version came out in Japan in 1999, helping spur the Japanese to make the earliest cell-cams. Motorola and Nokia ended up being late to the cell-cam game.

Cellphone cams evolved quickly. Most these days can take video as well as still photos. Kahn says he had some idea, even in 1997, that cell-cams would make a big impression.

“It wasn’t far from the Rodney King tapes,” he says, referring to the citizen-shot video of King being beaten by Los Angeles police officers. “It was clear that a little bit of videotaping had a massive impact on American culture. If you put that in the hands of a lot of people, and there are no barriers to sharing, it’s going to have a huge impact.”

We’re always watching

For the first time, hundreds of millions of people are carrying an image recording device all the time. It means somebody in a comedy club audience can see Michael Richards blow his wig and immediately capture it and post it on YouTube. The ubiquitous cell-cam seems particularly handy when some actress shows up having forgotten her underwear.

The always-there devices mean we get first-hand images of disasters, terrorist attacks and crimes. In his state-of-the-city address this month, New York Mayor Michael Bloomberg unveiled a program that will let citizens snap cell-cam photos of crimes and send them to 911. Hopefully, not many people will make use of PhotoShop software to etch a rival’s license plate number on a photo of an illegally parked car.

In 1984, George Orwell thought we’d be forced to behave because government cameras were always watching us. Instead, we’ll have to behave because every person is a spycam operator.

Cell-cam photos are the new autograph. See a celebrity, snap a picture and post it. The gadgets are recording innumerable tiny events that used to go into the ether — first baby steps, first kisses, first sales commissions. My 13-year-old son’s cellphone is filled with photos of fish he and his friends have caught in a nearby creek, most of them the size of a sausage link.

Kahn, who is working on a still-secret new company called Fullpower, altered society with his soldered-together contrivance. And the scary thing is, the infiltration of cell-cams is only beginning.

E-mail kmaney@usatoday.com
Original Article | Kevin Maney’s Blog

The Camera Phone

The gadget that perverts, vigilantes, and celebrity stalkers can all agree on.

Michael Agger – Slate

Ten years ago, Philippe Kahn was walking around a hospital with a cell phone and a digital camera. His dadly mission: to share pictures of his newborn baby girl. With an assist from Radio Shack, he linked the two devices together and e-mailed photos to family and friends around the world. The day marked a twin birth of sorts: the cell phone camera and daughter Sophie.

Kahn regards his invention with paternal pride: “I built it to document the birth of my daughter. For us, it has always been a positive thing.” So he was taken aback recently when, with the Saddam-hanging video circling the globe, an interviewer compared him to the inventor of the Kalashnikov. First there was Prince Harry’s Nazi costume, then the shaming of Kate Moss, then the Michael Richards racist explosion, but, for some, Saddam’s hanging marks the low point for Kahn’s creation. A camera on a phone has only aided the perverted, the nosy, the violent, and the bored.

That’s not exactly fair, but it’s not exactly wrong, either. As Kahn told Wired in 2000: “With this kind of device, you’re going to see the best and the worst of things.” The best would include photo caller-ID, amateur sports highlights, and the quick citizen snaps taken in the wake of the London bombings. Yet, despite the fun and occasional worthiness, the cell phone camera has launched a thousand jackasses. One representative example: Sportscaster Sean Salisbury was suspended by ESPN last month, reportedly for showing female co-workers cell phone photos of his “equipment.”

When video technology was added to phones (with little fanfare), the madness went up a notch. English youths devised a pleasant game called “happy slapping,” which involves assaulting random strangers while your mates record the whole thing. The happy slapping craze spread throughout Europe last year, leading to outraged op-eds and calls to ban cell phones from schools. While the phenomenon is marked by more than a touch of media hysteria, you can certainly find disturbing videos on YouTube. (The French, naturally, replied with “Streetkissing.”) There have also been news reports of graphic videos showing beatings and accidents, such as an unfortunate boy in Birmingham, United Kingdom, who impaled himself on his bicycle. Teenagers have employed cell phone cameras for old-fashioned humiliation, too: The parking lot fight is now captured on video and shared. To be an adult is to be grateful to have escaped the digital hazing of high school.

In glorious retrospect, it seems like a terrifically bad idea to give the world a spy camera that looks and functions like a cell phone. Peeping Toms quickly realized the potential for upskirt pics and shower-room souvenirs. Chicago tried to block cell phones from gyms, and a California legislator has proposed a law requiring the cell phone to make a shutter snapping sound or flash a light when a picture is taken. We have trained ourselves to be wary when a cell phone is pointed at us, but the device’s relative inconspicuousness still creates problems. In Saudi Arabia, women have been taking pictures of other women unveiled at weddings and e-mailing them to matchmakers, a practice that has caused uproar in a culture in which any sort of image can be cause for loss of honor.

The cell phone camera, constant companion, has also been championed as an anti-crime device. There have been several Rodney King moments, with bystanders pulling out their cameras to record sketchy police activity. One woman took a shot of a flasher on a New York subway, a photo that ended up on the cover of the New York Post the next day. There is also a mini-boom in sites to catch people who park like idiotsstare too long, and mistreat your kids. Think of this as the positive side of living in 1984.

The more difficult question, the one that lurks outside the media glare, is how the cell phone camera is altering our private lives. In the perceptive book Kodak and the Lens of Nostalgia, Nancy Martha West writes how Kodak, with the introduction of the personal camera, taught Americans to both conceive of their lives in terms of fondly remembered events and to edit out unpleasant memories. In Victorian America, for example, arranging to take a photo of a dead relative was not uncommon—a part of the grieving process. Under the reign of Kodak and its advertising, we became family historians of happiness. Now that digital cameras have taken over, the old photo album is giving way to the personal Flickr page, bringing with it a different set of assumptions of what to present (a whole lot more photos, for starters) and whom to share it with.

The ubiquity of the cell phone camera means that every moment in our lives is photographable. One consequence of this is an altered perception of the gravity of our day-to-day routines. We are now more aware of ourselves as observers of “history.” When a van catches fire in front of our house, we and our neighbors are now out on the lawn recording. We e-mail this to our friends, who testify to the enormity of the event, and then we all await the next sensation. This impulse can be positive, but it also fuels the increasingly destructive American habit of oversharing. The snapshot speaks with a small voice: I’m alive and I saw this. The cell phone camera picture or video is a shout from the rooftop: Check out this crazy thing that happened to me.

Picture sharing has also made us more aggressive in situations in which we feel insecure, such as in the presence of celebrities. Susan Sontag described the essentially hostile nature of taking pictures as a form of “soft” murder. In the age of cell phones, this scalp-hunting sensibility is achieving full flower. Let’s say you’re in Asbury Park and you see Bruce Springsteen with his kids. The old impulse would have been to ask the Boss if you could take your picture with him. The new impulse is to snap the shot with a cell phone camera and sell it to a site likeScoopt. No wonder famous people don’t want to hang out with us.

So, before we move on to the next racist comedian or cocaine-snorting supermodel, let’s put the Saddam video in context. It is a weird echo of the Zapruder film, another piece of amateur footage that caught the death of a leader. The differences are stark, of course. Zapruder captured Kennedy while standing openly in the Dallas sunlight. The official who videoed Saddam did so furtively, pointing his camera to the ground at times. But they both testify to the power of first-person witnessing, and how a digital copy of that witnessing can upend neat narratives and certainties. We’ll see the best of things, we’ll see the worst of things, we’ll see everything.

The Browser would like to thank the excellent Web site picturephoning.com.

michael.agger@gmail.com.
Original Article

NPR

This whole weekend, NPR (National Public radio) runs a big piece on the camera phone and its impact on society. I have to say that all business partners, friends and family members have had a great impact in shaping the camera phone. Click here to listen

Nanotechnology and micro machines

Nanotechnology and micro machines

The shrinking image sensors have allowed us to ship 1 billion camera phones in 2007. Now the same is happening with all sorts of other sensors. The key is MEMS technology, our ability to make mechanical and electromagnetic devices ever smaller. A digital compass the size of a needle-pin. Five gyroscopes that can be fitted in the form-factor of a dime. All this represents major technology breakthroughs. There is major innovation in building the hardware and now the software that makes it all work.

We are less than a decade away from the medical lab the size of a sugar cube.

We are less than a decade away from a truly non-invasive blood glucose and heart monitor.

We are close to the the next revolution in managing our world.

 

Happy Holidays

2006 has been a great year of Building technology.

Inventing.

Innovating.

Continuing to bring together the greatest core team.

Inventing the future is hard work. There is no feeling like that one. It’s like magic when you start seeing the technology really working.

2006 is winding down while even more excitement is building into 2007.

Happy Holidays.