Method and device for augmented reality message hiding and revealing
The present invention relates to an operation method for executing games on mobile camera enabled terminal devices and to devices capable of executing the method. The method is based on hiding augmented reality messages in pictures by taking an image of an object, providing symbols; linking the symbols to the image, such that the symbols are not visible when displaying the image, and sending the image and the linked invisible symbols via a wireless connection as a first image to a mobile camera terminal device. The invention also provides a second component to re-visualize the hidden message by receiving the first image, taking a second image with a camera of the mobile camera terminal device, displaying the second image, comparing the first and second images to determine a matching measure, and visibly displaying the linked invisible symbols on the display, if the comparison fulfills at least one predetermined condition.
Latest Nokia Corporation Patents:
The present invention relates to an operation method for executing games on mobile camera enabled terminal devices. The present invention may also be used for hiding messages in bitmaps. The present invention is also related to mobile games, messaging, and augmented reality.
STATE OF THE ART
The present invention refers to a game that is somewhat similar to a game called Geocaching. Geocaching is a “real life adventure game” for users of GPS (Global Positioning System) devices. The basic idea is that individuals set up caches all over the world and spread the locations of these caches via Internet. The GPS users can then use the coordinates of the caches for “rough navigation” to an area where a cache can be found. The word Geocaching is composed of GEO for geography, and CACHING for hiding a cache. Once found, a cache may provide the visitor with a wide variety of items as reward. Each visitor is asked to put something into the cache.
The cacheur (one who finds or looks for a cache) should also sign a visitors' book of the cache. Larger caches may consist of waterproof plastic buckets placed unobtrusively in the environment. The cache may contain the guest book and a number of more or less valuable items, which can turn the game into a true treasure hunt. The location of a cache may demonstrate the skill or even daring of the person who has laid out the cache. For example, a cache located on the side of a high bridge or a tree accessible only with rock climbing equipment may be hard to find.
Augmented reality is an area of virtual reality research. The basic idea behind augmented reality is to mix reality with virtual reality. So instead of generating an entire synthetic universe like in virtual reality, graphics, audio, and other sense enhancements are superimposed over a real-world environment to create an augmented reality environment. Augmented reality is closer to the real world than virtual reality because it adds elements to the natural world, as it exists. Augmented reality is sometimes also referred to as mixed reality.
Proposed application domains for augmented reality are, e.g., medical applications, entertainment, military training, engineering design, and consumer design. An example of a medical application is image-guided surgery, where, e.g., a surgeon can see magnetic resonance imaging (MRI) data correctly registered on the patient during an operation. A familiar example of augmented reality are television weather reports where the weather reporter seems to be standing in front of changing weather maps that are actually computer generated images. Augmented advertisements are sometimes used in TV broadcasts of, e.g., football games, where they are drawn virtually on top of special panels around the playing field. Also in ski jump broadcasts, typically one of the cameras also shows an augmented red line showing the length of leading jump. Augmented reality helped to the popularity of Americas Cup Sailboat racing contests to visualize the actual positions of boats in relation to each other and the wind on a plane water surface. Some gaming applications that use augmented reality are presented in the next chapter.
An augmented reality system can comprise of, e.g., the following elements: see through display, sensors that provide tracking information of user's limb- and, e.g., eye movements, haptics hardware that provides tactile feedback, audio hardware, and a small wearable central processing unit. See-through head mounted displays (HMDs) are often used and commercially available. However, an augmented system does not have to have all the components described here.
For example video games based on use of real world video or still images are known. The following games use augmented animations in connection with video streams and/or still image.
At the moment there are some applications that use augmented reality in games. Actual forms of augmented reality games are for example the game “REAL” (Reveal, Enclose, Appear Let's shoot) from tecmo, the game “Mosquitos” and the game “Attack of the Killer Virus” both from OJOM, and the game “Monsterhunt” from SysOpen Digia plc.
All these games are based on the “optical mouse principle”, by detecting motions of the background for directing a cursor (or in this case the background) on a see-through game board that is superimposed to the background image currently taken from a video camera of game phones. The game idea is based on a simple shooting game in the “real” environment while only in the display (scope/sight) of a device the targets (monsters, ghosts, virus, and mosquitos) are visible. In the display a crosshair is shown serving to aim a kind of “gun” at the targets.
Tecmo's game “Real” (2004) uses the idea that ghosts may be photographed. The player takes a picture and the game generates the shape of a ghost somewhere to the background of the picture. The player must then attempt to shoot the ghost through the camera.
In “Attack of the Killer Virus” and “Mosquitos” for Symbian S60 mobile phones these games may be interpreted as “Augmented Reality—shoot 'em ups”. In both games, the objects the player must shoot are superimposed on top of a live video stream from the camera of the mobile phone. Other similar games include at least “MonsterHunt” and “Agent V” from Digia Software for certain Nokia mobile phone models. All these games use live video stream to recognize how the phone is moved.
Another augmented reality game is the EyeToy from Sony used in connection with a Playstation 2. The EyeToy provides a kind of “fixed target suppression” using all components of successive frames with a changing signal as an event input. EyeToy also uses a see-through game board in which a player can see his movements in relation to the elements on the game area. Various “EyeToy” games have been very popular since they were first introduced in 2003. The player places himself in front of a television set and a small video camera connected to the game console. A picture of the player is shown on the television screen in real-time and placed “inside” the game environment through layered graphics. The game recognizes player's movements and these are used for controlling the game. There are several innovative “EyeToy” minigames: For instance, in “Keep Ups” game, the player must bounce a virtual football with his head and other body parts. In “AirGuitar”, a virtual electric guitar is superimposed in front of the player. The game recognizes the position of player's left hand on the virtual fingerboard and the movement of right hand strumming.
It is also known to use watermarking techniques in imaging.
All the known games provide video gaming in front of a real and currently video filmed background, relying on movements of or in the background as a main input component. Due to the similar game ideas these games are in danger of becoming only a kind of theme with variations of the same game idea, so that the following games such as “Cockroach Kill”, “Bedbug Blast”, “Gadfly Slay”, “Hornet Swat”, or “Jumping Snail !uash” may be expected in near future. Additionally, all these games suffer mainly from the same background in the game, i.e. the usual interior of a known environment such as the living room.
In all known games, there is actually no need for augmented reality, as there is no interaction between the content of the background (except, e.g., a white wall) and the game itself, basically all games just use the background only for recognizing scrolling movements.
Therefore, it is desirable to have a new game functionality that can be played on mobile camera telephones.
It is also desirable to have a new implementation of methods to execute or operate games on camera cell phones to promote user creativity and that may also serve as a reason for outdoor exercise.
It is therefore desirable to have a new way of using augmented reality in mobile applications for camera enabled devices such as, e.g., camera phones, smart cams, smart phones, PDAs and or smart navigation systems.
It is also desirable to provide new usage for mobile telephones as an additional feature to stimulate users to buy new cellular phones.
SUMMARY OF THE PRESENT INVENTION
The present invention deals with the idea to hide a text message in an image that is presented to the user in a game. The user has to find the place where the image was taken and take the same image with his camera or point the viewfinder in that location. If the viewfinder image matches the reference image, the text message is shown to the user.
According to a first aspect of the present invention, there is provided a method for augmented reality message hiding, comprising taking an image of an object, providing symbols and linking said symbols to said image, such that said symbols are not visible when displaying the image.
The said image and said linked invisible symbols are then sent via a wireless connection as a source image or a first image to a mobile camera terminal device.
The symbols may be in the simplest case a set of characters forming one or more words or sentences. Anyhow, the symbols may also comprise icons, signs, trademarks or other well-known graphic elements such as arrows that are able to indicate some kind of interpretable information. However, the symbols or graphic elements need not to be interpretable, so it may be possible to provide graffiti style ornaments to walls or the “usual decoration” like black teeth, moustaches, glasses, or horns to election posters or portraits of teachers, or the like.
By sending said image and said linked invisible symbols via a wireless connection as a source image to a mobile camera terminal device, a target with a hidden information may become available for, e.g., a game partner.
The invention describes a new way to hide messages in an augmented reality environment. A person selects some location or object, takes a photo of it, “hides” a text message to it using another image layer, and sends the original image and the hidden message to his friend using, e.g., a MMS message. The linking of the symbols to the image should be performed in a manner where it is difficult to extract the symbol in another way other than the intended way. The mission of the receiving person is to find that exact location or a similar object, and point to the location or object with the terminal camera. When the camera input is close enough to the original image, the hidden message is shown on the screen/display.
The game idea is that of a “paper chase” or that of a “scavenger hunt” without the necessity to actually lay traces. Paper chase is a children's game where a first group hides a first message on a starting point with a first route point, where the next piece of paper indicates the next route point. At the route points the traces are hidden so that the time to find the hint to the new route point approximately equalizes the time required to write and hide the piece of paper. The children's game scavenger hunt is based on a task list with the task to collect a number of (estimably ownerless) items in the environment such as, e.g., three leaves of a beech-tree, a crown cap, two acorns, a flat stone, a white round stone, a snail-shell, a piece of wire and a fir cone.
In an example embodiment said image and said linked invisible symbols are sent as a multi media message (MMS) via a cellular telephone communication network. That is, the first source image may be sent as an MMS via, e.g., a GSM network, or as an MMS via an UMTS network. It is also envisaged to transfer an executable program together with the first or source image to access said hidden symbols or message. It is also contemplated to use WLAN (wireless local area network) or WWAN (wireless wide area network) structures to transfer said first pictures, e.g., for teambuilding measures in Intranets. The invention may also be used to implement a “visual handbook” or “visual repair instructions”, for training mechanics.
In yet another example embodiment the method of the present invention further comprises defining regions of interest in said first image or source image, and linking information related to said regions of interest to said image. By using regions of interest “must match areas”, etc., and/or “must not match areas” (e.g., marked by a manufacturer logo, copyright sign or, e.g., an icon of an animal) may be implemented in the first picture. In every image/picture there are variable components, such as lighting, moving objects, such as animals, plants, people or, e.g., cars. It is possible to increase the functionality of the matching algorithm by excluding probably non-matching areas in the first image from the matching algorithm.
Another example embodiment of the method of the present invention further comprises preprocessing/post-processing, enhancement operations, color processing or, generating a virtual scene from said image taken. The taking of the image may include taking/recording a video stream or a number of images, pre-/post-processing, enhancement operations, and color processing. Generating a virtual scene from said image taken, especially the use of virtual scenes (e.g., in combination with a transferred hidden real image serving as the object for comparison), may reduce the possibility for cheating. The information a player needs is the position from which position and in which direction the image has been taken, this may also be achieved with a virtual scene. Virtual scenes have the additional advantage that the amount of data required to transfer the virtual scene may be drastically reduced (using restricted numbers colors or, e.g., data compression techniques).
Image processing may be used to compensate different lighting conditions that may pose a problem for the system. However, there are methods to calculate image similarity despite different lighting conditions, e.g., by using color-normalized images. It may be possible to use a number of images (taken, e.g., every half an hour during a 24 hour interval) and use region or contour matching only at the “all time light regions” or “all time visible regions” of a images, or of an averaged image, generated from a combination of the number of images. In such an embodiment the method would also comprise the steps of taking a number of images, and generating a source image from said images. However in a simpler version of the invention, the first image may just be a certain color in an “I spy with my little eye” fashion. “I spy with my little eye” is a children's game where one child states that it can see some object (another is probably not aware or) of a certain shape or a certain color. In the game another child tries to identify the object by questions that may more clearly define other properties of the object in question (such as, e.g., a number of feet). The game ends if a child can identify the object. In this case the lighting conditions do matter and the player must seek for proper light and color to be able to solve the task or make a match. Thus, it may also be an element of the method to force a participant to photograph the desired location as close to the original as possible such as the Sanctuary of the temple of Abu Simbel at morning of February 21 and October 21.
In yet another example embodiment of the present invention watermarking techniques are used to perform said linking of said symbols to said image in a way that said symbols are not visible when displaying the image. For example image watermarking techniques could be used to hide the symbols or the message. That is, the symbols, the message or the image containing the hidden message text is embedded as an invisible, robust watermark into the first or source image sent to the player. It is not possible or at least very difficult to detect an invisible watermark by hand. The watermarking may only be used in environments with a non-zero cheating probability.
The game application may be provided with a code to extract an embedded watermark. This could be implemented as a piece of program code in the game software. However, it may also be contemplated to use a dedicated firmware updateable watermark module. When the mobile camera terminal device is pointed at a supposed location, the mobile camera terminal device can recognize that there is a match, and will extract the watermark from the received reference image, and superimpose the extracted watermark image on the image from the camera.
Watermarks may be used to prevent cheating by presenting the camera, e.g., a print from the first image. A robust Watermark could even survive little modifications, so that when the mobile camera terminal device detects matching watermarks the original image may be deleted immediately.
Another example embodiment of said linking of said symbols to said image further comprises defining a position of said symbols in relation to said first image. It is envisaged to define also a font, a font size, a color a position and/or an orientation of the symbols in the first image. The position of the symbols may be defined in relation to certain pixels in the (first) image (in case a pixel matching algorithm is expected to be used for revealing the symbols), in relation to regions (in case a region matching algorithm is expected), in relation to shapes (in case a shape counting matching algorithm is expected to be used) or in relation to any combination thereof. This embodiment may also comprise the transmission of the defined fonts, font sizes, colors, positions, and/or orientations of the symbols (in a linked, encoded or non-coded manner).
According to another aspect of the present invention, a method for displaying hidden augmented reality messages on a mobile camera terminal device is presented. The method comprises receiving a first image or source image with linked invisible symbols via a wireless connection at a mobile camera terminal device. The receiver of the first image then (moves to a site where the object on the first image is located) takes a second image or a video stream with a camera of said mobile camera terminal device. The second image or the video stream is displayed on a display of said mobile camera terminal device. The mobile camera terminal device, then compares said first image with said second image. If the result of that comparison indicates that a match between the first and second images fulfills predetermined conditions, said linked invisible symbols are visibly displayed on the display of said mobile camera terminal device.
The received first image serves as a kind of target image or task image, such as the wanted posters in the “wild west” served as a job offer for bounty hunters. However, the first image is the “wanted poster”, the task is to photo shoot the object depicted, and the reward is the access to the hidden message.
In a simplest embodiment said, predetermined conditions are, e.g., a pixel match, wherein the first message with the revealed symbols is displayed on display.
In another example, embodiment the symbols are depicted with a “visibility” that is correlated to a matching measure between the first image and the second image. That is, in this embodiment the symbols are nearly invisible if the second image only slightly resembles the first image, and are clearly visible only if the second image strongly resembles the first image.
In yet another example, embodiment of the method of the present invention the comparing of said first and second images is performed by pixel matching, region matching, shape counting, or any combination thereof.
In just another example, embodiment of the present invention said visibly displayed linked symbols are displayed superimposed on said display in a fixed relation to said depicted second image, i.e. with a fixed relation to elements depicted on said second image. This should precisely read the visibly displayed linked symbols are displayed superimposed over the second image on said display in a fixed relation to at least significant objects in said first or source image recognized in said second image. This may be achieved by determining a perspective projection of the second image (or the 3D topography of the second image) to the first image (or its 3D topography) (or at least significant elements thereof), and a re-projection of the symbols to the second image. It may also be noted that the present invention may also use a projection module in the mobile camera terminal device to determine differences and the geometric cause of said differences between said first and second images.
It may also be possible to implement a camera position guidance system. The camera position guidance system may indicate to a user how to move the camera to reconcile the actual camera position with the camera position the first image has been taken with. The indication may be achieved by arrows depicted on the screen, or by a speech output that the user should move ½ meter to the left, lower the camera position, approach an object, and/or change the focal length of the mobile camera terminal device.
Another example embodiment of the method of the present invention further comprises sending a message that said linked invisible symbols have been visibly displayed, via a wireless connection from said mobile camera terminal device. The message may be sent to the sender of the first image.
According to yet another aspect of the invention, a software tool is provided comprising program code means for carrying out the method of the preceding description when said software tool is run on a computer or a network device.
According to another aspect of the present invention, a computer program product downloadable from a server to a readable medium for carrying out the method of the preceding description is provided, which comprises program code means for performing the steps of the client or the server of the preceding methods when said program is run on a computer or a network device.
According to yet another aspect of the invention, a computer program product is provided comprising program code means stored on a computer readable medium for carrying out the method steps of the preceding description, when said program product is run on a computer or a network device.
According to another aspect of the present invention, a computer data signal is provided. The computer data signal is embodied in a carrier wave and represents a program that makes the computer perform steps of the method contained in the preceding description, when said computer program is run on a computer, or a network device.
According to still another aspect of the invention, a mobile camera terminal device is provided. The mobile camera terminal device comprises a central processing unit, a radio interface, a storage, a display, a user input interface, an optical recording system, and a component to generate or display hidden augmented reality messages. The radio interface is for communicating via a communication network. The radio interface, the storage, the display, the user-input interface, the optical recording system, and the component to generate or display hidden augmented reality messages are all connected to said central processing unit.
The optical recording system can take first and/or second images. The user-input interface may be used to input or to provide the symbols to be hidden. The central processing unit may be configured to link symbols to taken images, such that said symbols are not visible when displaying the image on the display of the device or of another device. The display may also serve to display a second image actually taken. The radio interface may send said image and said linked invisible symbols via a wireless connection as a first image to another mobile camera terminal device. The radio interface may also receive a first image with linked invisible symbols via a wireless connection from another mobile camera terminal device.
With this configuration the device is capable of executing the methods for generating and displaying hidden augmented reality messages on said mobile camera terminal device.
The mobile camera terminal device may also be provided with a projection determination and control module, to determine the relative positions of the first and second images and to calculate the position (size orientation color . . . ) to the second image.
The mobile camera terminal device may also be provided with a dedicated module to provide symbols, as an alternative.
The radio interface may comprise a GSM UMTS module to send said image and said linked invisible symbols are sent as a multi media message via a cellular telephone communication network.
The mobile camera terminal device may also be provided with a dedicated region of interest defining module to be able to define regions of interest in said first image, and linking information related to said regions of interest to said image.
The mobile camera terminal device may also comprise an image-processing module for pre-/post-processing, enhancement operations, color processing, or generating a virtual scene from an image taken with the optical recording system.
A watermarking (hiding or encoding) module may be provided in the mobile camera terminal device to hide (or extract) the symbols in the first image by using watermarking techniques.
The mobile camera terminal device may also comprise an image positioning module to position and orient said symbols to said first or second image.
The central processing unit of the mobile camera terminal device may be configured to compare said first and second images. It is also contemplated to use a dedicated comparison module to compare the first and second images. The mobile camera terminal device can also comprise dedicated pixel matching modules, region-matching modules, or shape counting modules connected to the central processing unit to or incorporated in a dedicated comparison module.
The display may also display said linked invisible symbols on the display of said mobile camera terminal device.
The mobile camera terminal device may also be provided with a visibility module connected to said processing unit or to a comparison unit (and maybe to the display) to control the visibility of the symbols during visibly displaying said symbols in accordance with matching measure between the first image and the second image (e.g., provided from a comparing module).
The mobile camera terminal device may also comprise a superimposer to display visibly displayed linked symbols superimposed on said first or second images displayed on the display of the mobile camera terminal device.
The radio interface (or the central processing unit) may be configured to send a message that said linked invisible symbols have been visibly displayed on said mobile camera terminal device via a wireless connection to another mobile camera terminal device.
In another example embodiment, said mobile camera terminal device comprises a mobile telephone. That is, this embodiment is a cellular telephone that is enabled to execute the method of the present invention.
BRIEF DESCRIPTION OF THE FIGURES
In the following, the invention will be described in detail by referring to the enclosed drawings in which:
DETAILED DESCRIPTION OF THE FIGURES
In the detailed description that follows, identical or similar components have been given the same reference numerals, regardless of whether they are shown in different embodiments of the present invention. In order to clearly and concisely illustrate the present invention, the drawings may not necessarily be to scale and certain features may be shown in somewhat schematic form.
The principle of the invention of message hiding is depicted in
The hidden text may be regarded or may be visible, e.g., as an image layer on top of the original image. It may also be possible to hide the message/text in a layer located “behind” the picture. After this, person A sends the original image and the hidden message (as a first image) to at least one other person, e.g., his friend “Person B”. This could be done using, e.g., an MMS message.
When Person B receives the MMS message 30, he sees only the original image as shown in
If the camera input and the original image are totally different, the hidden message is not shown at all. If they are quite close, the message is inaccurate, too dark, or otherwise difficult to see from the terminal's screen. When the camera input and the original image are close enough, the hidden message (or a part of it) is shown on the screen, and Person B can read the message 36.
Person B could also write about finding the “cache” to the logbook just like in the case of Geocaching. Instead of a physical logbook, this could be done by sending, e.g., an SMS message to a certain telephone number or to a certain web page. In the case of a multi-player game implementation, different persons may try to find different messages and, e.g., solve a complex mission together. The hidden message could also contain an Internet address with a web-log where Person B could leave a message like in a logbook.
To make the game easier, Person A could give more detailed information about the object or location to Person B. In addition to the image and hidden message, for example text “This object can be found in Paris. It is a well-known landmark” could be sent to the receiving person. Other examples could include “This street is somewhere in Yli-Köyliö, Finland. Have a happy trip to the middle of nowhere!” and “The location of this train is a secret. If somebody finds it and reads the hidden message, he will win one thousand dollars from the Umbrella Corporation!”
Other applications are, e.g., a diary readable only when photographing a certain section of woodchip wallpaper or ingrain wallpaper. Other applications may reside in implementing a safe direction to a “letter box” as hidden and hiding place for storing operative information, Instructions, fake documents, technical and financial means etc. It is also possible to use the principle of the game as a kind of map or direction indicator, wherein all necessary information is displayed when a person is at the location. The present invention may also be used as a kind of independent tourist guide, showing additional cultural and historical information when a tourist is at the location of the landmark, building, memorial or monument of interest.
In simpler embodiments it may be possible to simply display the message in a part of the display of the mobile telephone. It is also possible to just display the whole first image with the visible augmented reality message in the display, which may be useful for reasons of power savings.
The analysis algorithms could be based on comparing features like edges and regions, average color of the whole image, certain regions, average brightness and contrast, correlation, and so on.
It may also be envisaged to perform an environmental three-dimensional scan by using an autofocus that scans all focal lengths and identifies the three dimensional structure from the regions of the images that are in focus.
A similar approach may be used to avoid mismatches between the pictures that are caused by different focal lengths of the device taking the first picture and the device taking the second picture. It may be possible to encode the focal length in the first picture. A three-dimensional environment with an objective having a focal length of 35 mm is quite different to the same scene taken with an objective having a 125-mm focal length. It is, for example, possible to use an IMEI (International Mobile Equipment Identificator) request in a game server to select a specific source image or “target image” taken with an optical device having a suitable focal length.
The simple system of
When it is determined if the second image matches the first image, various match-scoring schemes may be used. However it is also possible to implement organizer based games wherein a person or a number of persons are provided with a number of images to play public paper chases. The use of public events with a nearly unrestricted number of participants may serve to make the game public and known so that the paper chase feature has a chance of becoming a criterion for buying, e.g., a mobile telephone. A score may be derived directly from a kind of race type match. For rewarding, the scores can then be converted to monetary units or virtual credits used in many services. To boost the game, movie theatres or such may give out free tickets for the best players, in a paper chase game to all cinemas of town. A similar approach may be used for a paper chase to all fast food restaurants in town. In both cases a start image may indicate the starting point of the chase, and the revealed information may comprise a telephone number required to download a next first image, for a next target. In such arrangements it is also possible to send the challenge images out centrally, whereby they can be related to the event being promoted. A group of people may engage in a contest against each other according to a theme, and one of the registered players may win a free entrance or some other bonus, while the others just get the excitement.
The present invention can provide a new game functionality that may promote user creativity in mobile gaming and entertainment by opening infinitely many ways to modify and vary the gaming experience.
This invention can enable a very attractive gaming application for camera-equipped gaming terminals.
The invention can also be used for marketing purposes. A Company selling sports cars may send an image of a new, breathtaking sports car to its customers. By coming to the store and photographing the car, the customer could reveal hidden messages such as “Special offer: A free steep heavy metal CD for all buyers of this new car model!”
In the game course, cheating may be circumvented by a respective online implementation of the game. An online game has the additional advantage that a user may be charged for every online data transfer. Another advantage of an online game is that the text may only be transferred when a match is detected, that is there is no way to extract the “hidden text” from the first image. The game idea is a strict match/no match strategy. Therefore it could be envisaged to implement an “error” such as a stone, a gap, a crack an additional branch, a dog, or other “inadvertent” creatures or objects on a photo. If there is a match in the “inadvertent component,” that can indicate that the user used a cheat to obtain to the image. This may be implemented, for example, as a similarity threshold that should not be exceeded to score. The image may also contain a void area that is, e.g., grayed out, i.e., the image does not show all the image parts that have to match. However, in this embodiment the image the user has taken has to match with a reference image in all areas. Thus, a cheater would have to invent the grey area, if he did not take a picture on the correct site. This simple restriction may serve to filter a number of cheaters that may be rewarded with negative scores.
This application contains the description of implementations and embodiments of the present invention with the help of examples. It will be appreciated by a person skilled in the art that the present invention is not restricted to details of the embodiments presented above, and that the invention can also be implemented in another form without deviating from the characteristics of the invention. The embodiments presented above should be considered illustrative, but not restricting. Thus the possibilities of implementing and using the invention are only restricted by the enclosed claims. Consequently various options of implementing the invention as determined by the claims, including equivalent implementations, also belong to the scope of the invention.
1. A method comprising
- taking an image of an object,
- providing symbols;
- linking said symbols to said image, such that said symbols are embedded as an invisible watermark into said image, and
- sending said image and said linked invisible watermark via a wireless connection as a first image to a mobile camera terminal device,
- such that when the mobile camera terminal device is pointed at a second image, the watermark is extracted from the first image and displayed with a degree of visibility based on a degree of similarity of the second image to the first image.
2. The method according to claim 1, further comprising defining regions of interest in said first image, and linking information related to said regions of interest to said image.
3. The method according to claim 1, further comprising pre-/post-processing, enhancement operations, color processing, and/or generating a virtual scene from said image taken.
4. The method according to claim 1, wherein said linking of said symbols to said image further comprises defining a position of said symbols in relation to said first image.
5. A computer program product for executing a method capable of generating and/or reading hidden augmented reality messages, comprising program code sections stored on a machine-readable medium for carrying out the actions of claim 1, when said program product is run on a controller, processor-based device, a computer, a microprocessor based device, a terminal, a network device, a mobile terminal, or a mobile communication enabled terminal.
6. A software tool capable of generating and/or reading hidden augmented reality messages, comprising program portions stored on a readable medium, said program portions for carrying out the operations of claim 1, when said program is implemented in a computer program for being executed on a controller, processor-based device, a microprocessor based device, processing device, a terminal device, a network device, a mobile terminal, or a mobile communication enabled terminal.
7. An apparatus comprising
- a camera configured to take an image of an object,
- a processor configured to provide symbols and to link said symbols to said image, such that said symbols are embedded as an invisible watermark into said image, and
- a transmitter configured to send said image and said linked invisible watermark via a wireless connection as a first image to a mobile camera terminal device,
- such that when the mobile camera terminal device is pointed at a second image, the watermark is extracted from the first image and displayed with a degree of visibility based on a degree of similarity of the second image to the first image.
8. The method according to claim 1, wherein the extracted watermark is superimposed on the second image.
9. The method according to claim 1, wherein a code is provided for extracting the embedded watermark.
10. The method according to claim 1, wherein the second image comprises a particular location having a predetermined similarity to the first image.
11. The apparatus of claim 7, wherein the second image comprises a particular location having a predetermined similarity to the first image.
12. A method comprising
- taking an image of an object,
- providing symbols;
- linking said symbols to said image, such that said symbols are invisible when displaying the image, and
- sending said image and said linked invisible symbols via a wireless connection as a first image to a mobile camera terminal device,
- wherein during said linking said symbols to said image, an augmented reality message is hidden such that when the mobile camera terminal device is pointed at a second image, the augmented reality image is displayed with a degree of visibility based on a degree of similarity of the second image to the first image.
13. The method according to claim 12, wherein the degree of similarity of the second image to the first image is assigned a similarity score and the similarity score is used to determine the degree of visibility.
14. The method according to claim 13, wherein when the similarity score indicates that the first image and the second image match perfectly, the augmented reality image is completely visible.
15. The method according to claim 12, wherein display of the hidden augmented reality message is further based on the position and orientation of the augmented reality message hidden in the first image.
16. The method according to claim 15, wherein the position and orientation are used to project the hidden augmented reality message such that its position corresponds to the position it had in the first image.
17. The method according to claim 16, further comprising:
- displaying the augmented reality message at a predetermined location in the second image; and
- changing the degree of visibility of the augmented reality message as the mobile camera terminal device is moved.
U.S. Patent Documents
|5734752||March 31, 1998||Knox|
|6094483||July 25, 2000||Fridrich et al.|
|6356363||March 12, 2002||Cooper et al.|
|6504941||January 7, 2003||Wong|
|6704027||March 9, 2004||Nakano|
|6851052||February 1, 2005||Graveman|
|7002551||February 21, 2006||Azuma et al.|
|7095873||August 22, 2006||Venkatesan et al.|
|7337321||February 26, 2008||Terada et al.|
|7386159||June 10, 2008||Handley et al.|
|7663670||February 16, 2010||Orboubadian|
|7737965||June 15, 2010||Alter et al.|
|8090141||January 3, 2012||Eschbach et al.|
|8447067||May 21, 2013||Rhoads et al.|
|20030016842||January 23, 2003||Patton et al.|
|20030033530||February 13, 2003||Sharma et al.|
|20030068067||April 10, 2003||Fielding et al.|
|20030117584||June 26, 2003||Revelli et al.|
|20030191941||October 9, 2003||Terada et al.|
|20030231785||December 18, 2003||Rhoads et al.|
|20040001605||January 1, 2004||Venkatesan et al.|
|20040001606||January 1, 2004||Levy|
|20040051680||March 18, 2004||Azuma et al.|
|20040185885||September 23, 2004||Kock|
|20040250079||December 9, 2004||Kalker et al.|
|20050027600||February 3, 2005||Phillips|
|20050141750||June 30, 2005||Rhoads et al.|
|20050248668||November 10, 2005||Machida|
|20050251030||November 10, 2005||Azar et al.|
|20050259820||November 24, 2005||Jones|
|20050280720||December 22, 2005||Kwon et al.|
|20060115082||June 1, 2006||Kevenaar et al.|
|20060212705||September 21, 2006||Thommana et al.|
|20060215242||September 28, 2006||Besharat et al.|
|20060252455||November 9, 2006||Van Stuivenberg et al.|
|20070004461||January 4, 2007||Bathina et al.|
|20070035562||February 15, 2007||Azuma et al.|
|20070171288||July 26, 2007||Inoue et al.|
|20070177759||August 2, 2007||Eschbach et al.|
|20090293012||November 26, 2009||Alter et al.|
- www.geocaching.com; retrieved from the Internet on Nov. 2, 2005.
- http://0.tecmo.jp/index.shtml; retrieved from the Internet on Nov. 2, 2005.
- www.ojom.com/produces/mosquitos.html; retrieved from the Internet on Nov. 2, 2005.
- www.ojom.com/produces/killer—virus.html; retrieved from the Internet on Nov. 2, 2005.
- www.sysopendigia.com/C2256FEF0043E9C1/0/405000126; retrieved from the Internet on Nov. 2, 2005.
- www.eyetoy.com/index.asp; retrieved from the Internet on Nov. 2, 2005.
- “Robust DCT-SVD Domain Image Watermarkign for Copyright Protection: Embedding Data in All Frequencies;” A. Sverdlov et al; retrieved from the Internet (www.sci.brooklyn.cuny.edu/eskicioglu/papers/EUSIPCO2005-WM.pdf) on Nov. 2, 2005.
- Digital Watermarking: A Tutorial Review; S. Mohanty et al; retrieved from the Internet (www.cs.unt.edu/smohanty/research/Reports/MohantyWatermarkingSurvey1999.pdf) on Nov. 2, 2005.
Filed: Jul 29, 2005
Date of Patent: Jan 13, 2015
Patent Publication Number: 20070024527
Assignee: Nokia Corporation (Espoo)
Inventors: Mikko O. Heikkinen (Tampere), Kai Havukainen (Lämpäälä), Jukka Holm (Tampere), Antti Eronen (Tampere), Timo Kosonen (Tampere)
Primary Examiner: Claire X Pappas
Assistant Examiner: Afroza Chowdhury
Application Number: 11/194,129
International Classification: G09G 5/00 (20060101); G09G 5/373 (20060101); A63F 13/40 (20140101); A63F 13/30 (20140101); H04M 1/725 (20060101);