Chloe@Home (Leader: RWTH)
Global scenario description:
When Chloe returns from her sisters’ wedding party, the first thing Chloe does is uploading all the videos that she recorded with her HD video camera to the home media server. In the evening, she sits down in her sofa and watches the videos on her big screen television. As she starts to get hungry, she walks to the kitchen. The home system detects that she is moving based on the sensor which is built into her watch. As a result, the video is automatically migrated to the small TV in the kitchen.
After eating, Chloe calls her mother using her videophone. Since the call takes a long time, Chloe moves to different rooms while she is talking, and the session follows her on the different available devices (the videophone, the mobile phone, her multimedia station which is a video monitor with integrated microphone and camera). A Context System is running at Chloe's home server that provides localization information, making the migration process semi- or fully-automatic, depending on Chloe's preferences. If Chloe wishes to go out, the multimedia session can be carried out on her mobile device (smartphone, handheld) connected to some public network. Since her mother would also like to see the wedding video, Chloe gives access rights to her. However, since her mother has only a low resolution portable smartphone which is connected via a different network and has different capabilities, the video cannot be shown in its original high quality. As such, an adapted lower quality version of the video is needed which satisfies the mother’s user preferences. Chloe’s home server has an automatic annotation process, which detects the faces of people while uploading. The server steers the video adaptation process based on this information. The important parts of the video still maintain good quality, while the quality of the background is reduced. At some point, Chloe’s mother recognizes some friends in the video. However, as the display of the smartphone is tiny, she cannot see enough detail. She therefore adjusts the user preferences, such that only the regions of interest are zoomed and presented. However, since Chloe's mother is also hearing impaired, she has problems to understand the priest in the video. Thus, she activates her hearing profile and the audio stream is dynamically adapted to her impairment. The voice of the priest is now much better to understand.
NOTE: The scenario Chloe @ HOME is divided into 5 sub-scenario corresponding to 5 demonstrators
- Sub-Scenario 1 :Annotated video adaptation
Since Chloe’s mother would like to see the wedding video, Chloe gives access rights to her. However, since her mother has only a low-resolution portable smartphone, which is connected via a low bandwidth network and has limited capabilities, the video cannot be shown in its original high quality. As such, an adapted lower quality version of the video is needed which satisfies the mother’s user preferences. The server steers the video adaptation process based on this information. The important parts of the video still maintain good quality, while the quality of the background is reduced.
Demo Partners:
RWTH – Annotation tools that automatically detect [specific] objects (sub-scenario leader)
IBBT – Signal-level adaptation using annotated ROIs
ENST – Generation of a presentation layer for personalized multimedia rendering
Technical Innovations:
- Signal-level adaptation using annotated ROIs
- Annotation tools that automatically detect [specific] objects
- Generation of a presentation layer for personalized multimedia rendering
- Sub-Scenario 2 :Audio Adaptation
Chloe's mother is watching the wedding video. Due to recent degradation of her hearing abilities, it's difficult for her to understand the people in the video. Thus, the audio part of the video must be adapted to her hearing capabilities. Using the audio adaptation engine, Chloe's mother activates her audio adaptation profile. Her video player passes the audio stream to the audio adaptation engine and this engine dynamically filters the audio stream according to Chloe's mother's hearing capabilities. Thereby, the speech intelligibility of the video is improved.
Demo Partners:
OFFIS – Signal level audio adaptation and creation of a hearing ability profile (sub-scenario leader)
GET-ENST – Video playback and audio handover
Technical Innovations:
- Signal level audio adaptation to increase understandability
- Creation of a hearing ability profile
- Video playback and audio handover
- Sub-Scenario 3 :Session migration
Chloe is reviewing her wedding movie on some intelligent device (for example, a general purpose desktop PC / laptop or a next-generation TV with integrated communication facilities). At some point in time, Chloe receives a call from her mother, who is eager to know the news about the marriage. She answers the call and starts talking to her mother. The conversation takes a long time, but Chloe has a lot of work to do; she has to clean the kitchen, she has to put her wedding cloth back into the furniture, and other things. While talking, Chloe moves to different rooms. A context system continuously tracks her position, so that the active session always follows her on the closest device. Once she has finished her work, it's time to go out: she has to go to her university. Chloe takes her mobile phone, and the session is migrated to the portable device. While at home the smartphone connects through the high-bandwidth and low-cost WiFi network, it now switches to the public cellular network. The portable device optimizes the network connection based on Chloe's preferences stored in her profile (the selection process may be driven by bandwidth, delay, availability, stability, quality, security, and cost as the main metrics).
Demo Partners:
UNIGE – Session migration for real-time interactive multimedia communications (sub-scenario leader)
ICOM – User localization
CNR – Abstraction and Integration of heterogeneous context information by means of distributed Context Server
GET-ENST – Multimedia rendering
Technical Innovations:
- Session migration for real-time interactive multimedia communications:
-vertical and horizontal handover
-session migration between heterogeneous devices
-context information
- User localization
- Abstraction and Integration of heterogeneous context information by means of distributed Context Server
- Profile information
-Media adaptation, device capabilities, network conditions, and user preferences
- Multimedia rendering
- Sub-Scenario 4 :Remote Access to the home content
Chloe is reviewing her wedding movie on some intelligent device (for example, a general purpose desktop PC / laptop or a next-generation TV with integrated communication facilities). At some point in time, Chloe receives a call from her mother, who is eager to know the news about the marriage. She answers the call and starts talking to her mother. The conversation takes a long time, but Chloe has a lot of work to do; she has to clean the kitchen, she has to put her wedding cloth back into the furniture, and other things. While talking, Chloe moves to different rooms. A context system continuously tracks her position, so that the active session always follows her on the closest device. Once she has finished her work, it's time to go out: she has to go to her university. Chloe takes her mobile phone, and the session is migrated to the portable device. While at home the smartphone connects through the high-bandwidth and low-cost WiFi network, it now switches to the public cellular network. The portable device optimizes the network connection based on Chloe's preferences stored in her profile (the selection process may be driven by bandwidth, delay, availability, stability, quality, security, and cost as the main metrics).
Demo Partners:
TELEFONICA – The remote access of multimedia content stored at home from anywhere (sub-scenario leader)
UNIKLU – meta-data based content search
CU – profile management
Technical Innovations:
- The remote access module allows a user to access all her/his multimedia content stored at home from anywhere, not having to worry about the NAT traversal or firewall issues. This solution breaks the barrier existing between the home and the external network in a secure way.
- The UPnP protocol allows a user to easily find all multimedia content stored at home and play it in any available device at home. With the remote access module this is extensive to UPnP devices outside one’s home.
- Furthermore, the metadata allows a user to improve the content’s search as she/he can browse or search for multimedia content by any of the numerous supported metadata fields, such as actors/actress, year of creation, involved locations, keywords, genres, ratings, etc.
- Sub-Scenario 5 :Animated picture presentation
During the family’s major holidays in the UK, the members of the family try out a new feature provided by the greater museums they plan to go to. Chloe and her husband cannot spend much time in the huge exhibitions since the children’s interest and patience for arts is limited. They are glad about the new multimedia art guides: Before deciding where to go next in a museum, they can connect a mobile phone to the museum’s network and download animations for the different exhibition. Each animation covers the highlight paintings of an exhibition. A virtual camera zooms and pans over the paintings, switching between overviews and details, focusing particularly on objects that are referred to in the descriptive text. The animations are adapted to the time that users want to spend for viewing, their interests (if available), and the display size. The impressions of what to expect in each exhibition is more vivid and effective than any brochure could be. Chloe and her family are able to get the best out of their limited time for the many great exhibitions. Back home, the animations still make great souvenirs and screensavers.
Demo Partners:
UNIKLU – Extracting references to objects in the image from text; transmoding (image to video) adaptation for large pictures, preserving experience under restricted conditions (sub-scenario leader)
GET-ENST – playback component for animations
Technical Innovations:
- Extracting references to objects in the image from text
- Satisfying user’s constraints and preferences (available time, resolution, camera motion speed restrictions)
- Transmoding (image to video) adaptation for images for preserving experience (zoomed views of relevant details and scaled overviews connected by smooth motions and pauses where appropriate) under restricted conditions, based upon matching of image features and analysis of textual descriptions, with respect to user’s preferences