A Survey of Latest Performance, Development and Measurement Issues of Smart Phones Design

Shengkui Gao, gaoshengkui@wustl.edu (A paper written under the guidance of Prof. Raj Jain)

Download

Abstract:

Keywords: smart phone, performance evaluation, measurement, user experience, RRC, 3G, directional antenna, keypad, GPS, Visual interaction, VoIP, Wi-Fi, code-offload, mobile security, smart phone OS, smart phone usage, middleware, cross-cultural factor.

Table of Contents

• 1. Introduction
• 2. Hardware Performance
  • 2.1 Network Performance
    • 2.1.1 Network carrier performance
    • 2.1.2 Radio resources allocation performance
  • 2.2 Antenna diversity performance
  • 2.3 Keypad input performance
• 3. Additional Modules Performance
  • 3.1. GPS module performance
  • 3.2. Camera module performance
  • 3.3. Wi-Fi data communication
• 4. Software Performance
  • 4.1. Code offload performance
  • 4.2. Security Performance
• 5. System Performance
  • 5.1 Phone tasks performance
    • 5.1.1 Cross-platforms phone tasks performance
    • 5.1.2 Different tasks modeling on smart phones
  • 5.2 Smart phones middleware development and performance
• 6. Smart Phones Usage Performance
  • 6.1. Usage diversity of smart phones
  • 6.2. Cultural Differences evaluation of smart phones
• 7. Summary
• 8. References
• 9. List of Acronyms

1. Introduction

Smart phones are becoming more and more popular in recent years due to its multiple functions, portability and easy to access anywhere. Accompanying with the development of 3G and 4G communication network, the smart phone is gradually replacing the regular cell phone. In 2009, 14% of the worldwide mobile phone shipments were smart phones [CanalysPress], and it's predicted to surpass the desktop PC sales by 2011. The performance analysis and measurement of smart phones are becoming a hot research topic in mobile computing research society. However, since the smart phone only becomes as a commercial product recently and many aspects still need to be further studied. Moreover, the smart phone configuration and specs are relatively confidential among different cell phone producers. Therefore, there are quite few public performance standards existing for smart phone performance evaluation.

Some main stream smart phone specs are shown in Table 1. Since the main specs of the smart phone such as CPU speed, memory size, I/O specs in hardware domain, code efficiency, function realization in software domain and mobile system design have been studied by following the similar evaluation metrics and methods used in computer design. But in normal smart phone use, the advanced smart phone may not perform well due to the constraints such as network, battery power and used applications. Therefore, this paper will focus on the special characteristics performance and their impacts on smart phone, and also smart phone usage performance.

In this paper, some latest design progress and new evaluation methods for different parts of smart phones are being briefly introduced. This paper is organized as follows: Session 2 introduces the hardware (network, radio resource, directional antenna) performance and evaluation metrics of smart phone. Session 3 introduces additional modules (GPS, camera and Wi-Fi) performance; Session 4 discusses about the software performance issues (code offload and mobile security); Session 5 states the system level (different platforms, different task modeling and middleware design) performance and evaluation methods; Session 6 discusses the usage (diversity of the usage and cross-cultural factor on smart phone usability) performance of the smart phone. And a brief summary of the smart phone performance is given in the Session 7.

2. HARDWARE PERFORMANCE

The hardware design of smart phone is a very important aspect due to its size limitation and battery life. The core parts of smart phone such as CPU, memory and I/O peripheral are used the similar performance analysis methods as used in computer evaluation, power and speed are the key metrics. Furthermore, there are other special components and constraints in the smart phone hardware design, such as cellular network performance, antenna configuration, keypad style, which are going to be stated below.

2.1 Network performance

The cellular communication network is the most important aspect to guarantee the smart phone realizing most of its functions. Different network operators and different radio resources could greatly influence the performance of smart phones.

2.1.1 Network carrier performance

Different network carrier can offer different network performance for the smart phones, even the same type of phone used in different networks can performs differently. Since the main function of smart phone is surfing internet and getting network based services, the main evaluation object for smart phone network performance will be the data network. Recent published papers of network performance evaluation focused on understanding the network behavior for cross-layer measurement [Mattar07], multi-rate multi-user measurement [Ghaderi07] and multi cellular network carriers' measurement [Huang10].

Major parameters used in network performance analysis are: downlink/uplink, time of day, signal strength.

The main metrics considered on the network side are: Throughput, round trip time (RTT), retransmission rate, local DNS lookup time, handshake time and Ping latency.

Different types of network service workloads are: web browsing, email service, stream video and voice over internet protocol (VoIP) communication, etc.

Several cellular network measurement tools are currently available online, like 3GTest [Huang10], Speedtest.net [Speedtest] and FCC's broadband test [FCCBroadbandtest] etc. And current results show the real 3G network performance is much lower than the theoretical limitation, so large amount of work is still needed to improve current 3G network performance.

2.1.2 Radio resources allocation performance

In current 3G communication network, the radio resource control (RRC) state mechanism for smart phone could cause large performance inefficiency due to state promotion overhead. The tradeoff between saving battery power and quick network response is critical for smart phone design. Recent work are focusing on discussing the impacts of inactivity timers for network performance [Chuah02], the impact of different timer values on energy consumption and network service quality [Liers07] and dynamic RRC control optimization for 3G network [Qian10].

Figure 1: Basic RRC transmission states sample diagrams of different carriers. [Qian10]

The RRC for current smart phones (Figure 1) mainly have three states: IDLE: default state when phone turns on. (No radio power consumption, no bandwidth) Cell_DCH: the connection is established and data is in transmission. (High power consumption, high bandwidth) Cell_FACH: on connection but no data is in transmission. (Low power consumption, low bandwidth)

Different carriers offer different strategies to optimize these three states and apply for different network services (web browsing, streaming video, etc). Downlink/Uplink difference, State promotion overhead and tail effects should also be considered.

Main metrics used to evaluate radio resource allocation are: interaction time delay for state transitions, power consumption for sending certain size of data, throughput, the time ratio of Cell_DCH and Cell_FACH.

Classic workloads for testing are: email service, map data, streaming video and streaming audio.

Current experiment results RRC optimization could bring great improvement of energy saving and high service quality, and dynamic RRC control could perform better in saving power consumption without losing too much speed performance than the existing fixed state RRC control.

2.2 Antenna diversity performance of smart phones

Good antenna design could bring great power savings and high network performance to the smart phone. Right now most of smart phones are still using Omni directional antenna, which is power waste and can also introduce interference among peers. Some researchers are focusing on improving antenna performance by applying two or more Omni-directional antennas for diversity [Norklit01]. The new multi-directional antenna system [Sani10], which uses accelerometer and compass (commonly existing in current smart phones) to assist orientation estimation, are being studied in order to achieve good signal strength, goodput, and further power savings.

A complete smart phone antenna evaluation should consider different environment factors: outdoor/ line-of-sight indoor/non-line-of-sight indoor and different signal strength.

Different parameters of antenna: The number of directional antennas being used, antenna configuration positions, Different signal noise ratio and the modulation styles

The workload is constant size of data packets.

The simulation results show that multi-directional antenna system could achieve better signal strength, goodput and lower power than Omni-directional antenna. Moreover, the antenna performance can be effectively improved by using accelerometer and compass assistance for orientation estimation, and then based on the orientation information result select the corresponding combined signal of directional antennas.

2.3 Keypad input performance

The styles of the keypads for smart phone have several different categories: traditional digital keypad, Qwerty keypad and on-screen keypad. The different keypads style could greatly influence the user experience for the smart phone. In order to increase the input speed of keypads and make users feel more comfortable to use smart phone. Smart phone designers have designed many input methods and add user preference memory function to the smart phone. Recent researches of keypad evaluation are focusing on using statistical models and emulation to do keypads evaluation, such as using Keystroke-Level-Model to decompose task into primitive operations and predict completion time [Holleis07], paper mockups emulation of keypad layout [MacKenzie07] and TnToolkit evaluating keystrokes per character (KSPC), word per minute (WPM) [Castellucci09].

Main metrics for keypads evaluation are: complete time, error rate of tapping, language-based disambiguation, KSPC [MacKenzie02] (the average number of keystrokes required for a single character of text using given interaction technique in give language) and WPM performance prediction [Silfverberg00] (predict movement time by considering button distance and button width).

Common workload for keypad is usually constant size of text, or some text benchmark such as KSPC26 and KSPC27 [MacKenzie02].

The workload is constant size of data packets.

The smart phone keypad is always a bottleneck for working efficiency of mobile devices comparing with desktop PCs due to its area and less number of keys. More and more tools and models are developing for evaluating and improving current keypad styles, and we believe better keypads will dramatically increase the input efficiency for the smart phones.

3. ADDITIONAL MODULES PERFORMANCE

Comparing with normal mobile phones, other than fully used the network advantage to do data communication, smart phones also have many additional modules such GPS, camera and Wi-Fi, which could help smart phones to possess new functions and improve performance.

3.1 GPS module performance

The GPS module can not only make the smart phone to become a GPS navigator, but it can also help the smart phone to provide location-based and context-aware services. Though the navigation function could also be realized by GSM/Wi-Fi based positioning system, the GPS can supply more precise information. The disadvantage of GPS is high power consumption. So a better GPS control mechanism and combined use multiple different localization methods could achieve a large power saving for the smart phone.

Recent GPS based localization and power tradeoff research for mobile devices is quite hot. Several ongoing research projects are listed: RAPS: A rate-adaptive positioning system for smart phone applications are presented by using GPS as often as necessary to achieve accuracy and reduce power consumption, using accelerometer on smart phone as motion reference and Bluetooth communication to reduce uncertainty among neighboring devices [Paek10]. Enloc: dynamic programming for finding optimal accuracy under given energy limitation among GPS/GSM/WiFi [Constandache09]. A-Loc: using probabilistic models to dynamically balance accuracy and energy use by choosing among different localization methods [Lin10]. A location-sensing frame with accelerometer based suppression, location-sensing piggybacking, substitution of location-sensing mechanism and adaptation of sensing parameters during low battery to increase power efficiency [Zhuang10].

Main parameters considered in GPS performance evaluation are: Different weather conditions (large influence on GPS signal strength), outdoor/indoor, object movement (still/ low/ high speed).

Main metrics used for GPS evaluation are: power, distance error and positioning time interval.

Due to the researchers' different areas, no common workload or benchmarks for GPS testing, common workloads are certain routines near the researchers' working place.

Due to large power consumption of GPS, current researches are focusing on the dynamic usage of GPS/GSM/Wi-Fi services based on the accuracy requirement, use other sensor like accelerometer to provide motion information for positioning interval prediction and other communication method like Bluetooth to reduce positioning uncertainty among neighboring devices [Paek10], which can provide smart phone an efficient way to supply positioning information at low power level without losing too much precision.

3.2 Camera module performance

The camera is one important part of the smart phone. It can not only capture pictures, but also provide support for control or data collection of other applications in the smart phone system, like motion detection, face reorganization, visual input for smart phone control, etc. Due the power limitation of the phone battery, the imaging sensor with large number of pixels and high visual quality is hard to implement in a smart phone. Therefore, in current smart phone camera research, how to use vision based interaction to increase current camera capture efficiency is focused by many researchers. They try to optimize the image results and increase process speed is another way to improve camera performance for smart phone. Different programming languages (C++, Python and Java, etc) are evaluated under the same smart phone operating system (OS) system.

Several metrics in image processing domain are applied to evaluate their performance: Frame per second: The number of full size bitmap frames captured from the camera and then saved to memory in one second. Read latency: the time interval between picture captured and shown on screen. Pixels skip n: image de-sampling processing. Time to track: the time track a small object from the background (example: a red dot in a white background) in the center of view finder.

The workloads are usually constant number of images with certain frame rates.

Several commercial benchmarks are available for smart phone image processing, game performance, such as SPMark [SPMark11], 3DMark Mobile, etc.

The experiment results show different programming languages have quite different performances. Better memory controlled and lower level language, like C++, has better speed, but less deploy-ability, for Java, though it has lower speed, but higher deploy-ability [Kerr09].

3.3 Wi-Fi data communication

The Wi-Fi network access is another important function for smart phone, because it could offer higher bandwidth and lower response latency comparing with cellular network. However, since Wi-Fi doesn't have efficient RRC mechanism for smart phone, it could be power hungry in default mode for data communication and VoIP using Wi-Fi. Designing a better Wi-Fi control mechanism is critical for smart phone design. Moreover, though Wi-Fi is quite power consuming in the default mode, it consumes less power when doing VoIP communication than normal phone call using GSM/3G network. So mixing control for Wi-Fi and normal GSM/3G operation could provide a large power save. Some recent researches discuss the energy-delay tradeoff by seeking power saving for both Wi-Fi and GSM/3G network while maintaining the data transmission efficiency [Ra10], and Cell2Notify: Wi-Fi wakeup mechanism to better manage the power consumption [Agarwal07].

Main parameters considered in Wi-Fi performance are: different signal strength, different object movement (still/low/ high speed).

Metrics mainly used in Wi-Fi function evaluation of smart phone are: power, battery lifetime, connection latency, failure rate and bandwidth.

Common workloads for Wi-Fi are: certain size of data packets, stream video and VoIP communication, etc.

The experiments results show by optimizing the energy-delay tradeoff, the smart phone could save 10%-40% power [Ra10], and applying wakeup control mechanism for Wi-Fi to do VoIP or data communication could greatly save power for smart phone by a factor from 1.7 to 6.4 [Agarwal07].

4. SOFTWARE PERFORMANCE

The software efficiency for smart phone largely influences the execution speed and power consumption of the smart phone. Two main software aspects are discussed below: the code offload mechanism and security performance smart phones.

4.1 Code offload performance

Code offload is one popular technique used to reduce power consumption, break through the hardware constraints (CPU, memory) of the smart phone and achieve high speed for the task execution of smart phone. It divides executing tasks into two categories: For small computation required and less power consumed task, it will execute locally in the smart phone; For large computation required and heavy power consumed task, it will take advantage of delegating code at rich resourced remote server side to increase execution speed and save battery power for the smart phone. But the latency of the communication between the server and the smart phone and the power cost by the communication and synthesizing results need to be considered.

Figure 2. Code offload mechanism diagram

A basic diagram of code offload mechanism is shown in Figure 2. The smart phone side applications send requirements, tasks to the server, then server finishes the tasks and sends back results to the smart phone. Recent research in code offload are focusing on: using code offload to achieve power saving and high speed [Cuervo10], or break the hardware limitation on smart phone [Gu03].

System parameters considered in code execution are: Different types of network connection (Wi-Fi, 3G), code size, task types (document processing, video game, image processing)

Main metrics used for evaluation are: power, frame per second (used in game and image processing), data throughput.

Common workloads for code offload evaluation are: image processing files (video games), data computation (chess) and database based work (face recognition, etc)

The simulation results show code offload can reduce the power consumption of the smart phone and increase the execution efficiency for heavy computation required task. But the strategy of network connection, task profile standards, synchronization between server/user device and code transmission protocols are critical for code offload application in smart phone design [Cuervo10].

4.2 Security Performance

Since the smart phones are widely used in personal data service (email, shopping, bank service), company business and government work, the security issue has became an important issue for smart phone design. Many security standards has been published to protect web safety and message encryption, such as WS-Security [IBM02], SAML [IBM02] and LA [LA06].

The performance of different security standards are mainly evaluated after applying a constant length of SOAP (a XML based transport protocol) or HTTP data, the data packet size differences among different encryption standards and their data transmission latency [Srirama07].

Due the limitation of smart phone hardware and its power, the security standard for smart phone is still under developed, and it's especially lack of evaluation benchmarks and standards for different security standards performance in protecting information and detecting phone virus or spam.

5. SYSTEM PERFORMANCE

Due to commercial reason and cell phone development history, the smart phone OS is quite isolated. So different smart phone OSs show large performance differences even when they execute the same task. It's important to evaluate system performance of smart phones for better designing future smart phone OS. The phone task organization and middleware design are two important aspects for smart phone system performance.

5.1 Phone tasks performance

Today's different smart phones can execute almost the same tasks like phone call, messaging, email, web service and gaming, etc. But they show dramatically different performance due to the different platforms they base on and different task models they use.

5.1.1 Cross-platforms phone tasks performance

Current smart phones are based on three main different platform types: cell phone type platform (Nokia), PDA type platform (Palm, Blackberry) and PC type platform (Windows Mobile, Mac iOS, Android). They perform differently when executing the same type of task. Some recent researches evaluated the different types of tasks on different smart phone platforms [Keijzers08].

Main tasks considered to evaluate different platform performance are: 1) Phone task: go to contacts to search certain people and make phone call. 2) Internet task: browse certain website. 3) Calendar task: make certain memo and set up alarms. 4) Mail task: receive and reply an email.

The main parameters considered while evaluating performance are: user's age, careers, past experience of using the smart phone.

The metrics used to examine the performance are: time cost, number of menus used, number of detour steps and user satisfaction.

The current experiment results show none of different platform systems can win all tasks. In general, cell phone type platform performs better in phone task and calendar task. PDA type platform works better when doing push mail task. And PC types platform perform better when used to do internet task and calendar task [Keijzers08].

5.1.2 Different tasks modeling on smart phones

One big convenience of the smart phone is that it can do some tasks that people used to finish only on a PC. But due to the hardware limitation and undeveloped smart phone OS and application program. People always feel trouble using smart phone to finish certain type of job instead of using PC. Identify the different task modeling on smart phones and the poor performed task of smart phone could help the designers to improve smart phone in future. Recent works in this area focus on investigating people's manner of accessing information from laptop and smart phone [Dearman08], difficulties of managing data from various devices [Oulasvirta07] and the task accessing difficulties among smart phones of different Oss and PC [Karlson10].

The basic idea for testing task modeling for smart phones is to do the same task under different smart phone Oss (Windows Mobile, Mac OS X and Android, etc), then compare the difference of user experiences, users take snapshots or write comments when they have trouble while executing certain task.

Main tasks and workloads considered for evaluation are: email, web surfing, maps, calendar, social networking (MSN, Gtalk, etc), file management and multi-media playing.

Based on analyzing the snapshots and user's answer of certain questions, the task usage problems under different smart phone OSs show the in general the similar experiences, which means current smart phone OSs still need to be further developed to satisfy the users daily requirement.The metrics used to examine the performance are: time cost, number of menus used, number of detour steps and user satisfaction.

5.2 Smart phones middleware development and performance

Middleware is the control management part between the OS/external sensors/wireless communication and specific applications, which is shown in Figure 3. It offers expansibility and compatibility for the smart phone to act as a pervasive computing device, which could be widely used in scientific research, medical care and daily life. Due to the isolation of smart phone OS and limited application programming interfaces (API) it supplies to the application developers, the traditional applications could not achieve the best performance and even poor power management when applied to the smart phone. Therefore, recent middleware researches focus on improve power consumption performance and optimize application execution efficiency. Some featured researches are: Fuego Core: using XML messaging, synchronization and mobile distributed event-based communication for smart phone communicating with other equipment [Riva08]; CAMPH: context-aware middleware for pervasive homecare [Pung09].

Figure 3. Middleware structure of smart phone

Main metrics are considered in middleware design are: data communication specs (throughput, latency), power consumption, execution speed for different programming languages and OS platforms, the precision per recall.

Current research results show building up a generic middleware layer between the smart phone OS/external sensors and applications can greatly save power and increase the application execution efficiency while using smart phone as a pervasive computing device, and it also provide the portability for pervasive computing. The context-aware mechanism can offer good querying processing and accurate activity recognition.

6. USAGE PERFORMANCE

As a daily used equipment, the usage of smart phone is different among different groups of people and people from different parts of the world. The smart phone could perform poor to a user if it cannot satisfy the requirement of this specific user. Therefore, the usage diversity and cultural differences evaluation are important for smart phone design and performance.

6.1 Usage diversity of smart phones

Different people use smart phones to accomplish different tasks. The capability of the smart phone to adapt to user behavior could largely affect the smart phone performance and user experience. Comparing the users' interaction activities with the smart phone can provide the smart phone designer more information on how to improve the smart phone configuration and its OS. Some recent works focus on analyze user's behavior using statistical models for task diversity usages [Falaki10] and battery consumption characteristics [Shye09], etc.

System parameters considered to evaluate smart phone usage are: user's age (youths, mid-age knowledge workers, etc), career (social communicators, life power user, business power user, practical organizers, etc), time of day and user percentile.

The main metrics used to evaluate smart phone usage are: 1) User interaction (session time and number of sessions). 2) Application use (different types, time and number of sessions). 3) Network traffic (different application related (web browsing, online game, streaming video, etc ) data transmitted). 4) Energy drains (Battery drain).

The workload is usually daily usage of the smart phone.

Based on the evaluation results, the application popularity for users is approximately follow the exponential distribution [Falaki10]. But the popular applications for different types of users are largely different, so the usual smart phone design mechanisms that is only based on average case could not be effective for common use, the smart phone designer need to focus on the improving the smart phone to be more adaptive and learn from the user behavior.

6.2 Cultural Differences evaluation of smart phones

Cross-cultural factors have a great influence on how user experiences of people from different countries are. The smart phone usability performs different under different cultural environment. The main impacts of cultural differences can be divided into two categories: Objective issues (language, dates and number format convention, text directionality) and subjective issues (value systems, behavioral and intellectual systems, and the cultural interactions with computers and websites) [Walsh10].

The main analysis method for cross-cultural usage of smart phones is using online survey and let user complete certain description under different environments. The Hofstede's culture dimensions are commonly used to analyze cultural differences [Schumacher10]. It has five metrics:

1. Power distance: the social expect from people of different classes.
2. Uncertainty: how people feel when face ambiguity and risk, or big change.
3. Individualism: how people want to be different with others or get involved which social group.
4. Masculinity: how people deal with different issues.
5. Long term orientation: how the societies embrace or not embrace long-term devotion to tradition values.

The survey questions should be developed based on these five Hofstede's metrics, and cover usability, aesthetic aspect, subjective feeling, identification and brand/image [Walsh10]. Based on the survey results, different cultural people do have different preference on smart phones usability and performance evaluation. Even for the same model product, people from different countries usually give different evaluations. Doing a detail research of cross-cultural factor can provide a good reference for smart phone manufacturer to design their product for specific regions.

Based on the evaluation results, the application popularity for users is approximately follow the exponential distribution [Falaki10]. But the popular applications for different types of users are largely different, so the usual smart phone design mechanisms that is only based on average case could not be effective for common use, the smart phone designer need to focus on the improving the smart phone to be more adaptive and learn from the user behavior.

7. SUMMARY

In this paper, the latest development, performance evaluation and measurement methods for different aspects (hardware, additional modules, software, system and usage) of the smart phone are introduced. Though different aspects of the smart phone may have different metrics and parameters to consider, accomplishing the high power and performance efficiency and the better user experience are the two common design goals for all the aspects of smart phone design. Several observations could also be made:

1. In the hardware aspect, network performance and power optimization of the smart phone could be achieved by applying more advanced RRC mechanism and multi-directional antenna.
2. In the additional modules aspect, different modules could supply rich additional functions for better smart phone performance, but the power consumption becomes the key factor due to the power limitation of smart phone battery.
3. In the software aspect, the code offload could provide a new power saving way and increase the code execution efficiency. And the security performance goal is to achieve the top safety with the minimum size packets.
4. In the system aspect, better task arrangement mechanism and middleware design could increase the user experience and achieve higher power save.
5. In the usage aspect, fully considering the user's identity and cross-cultural factors are important when designing the world popular smart phone.

Currently, the smart phone design is still a new research area that lacks of systematic performance evaluation method and standards. Many papers referenced in this paper are reported as the first work in the corresponding domain. But many components of the smart phone, like GPS, Wi-Fi, smart phone usage and OS task evaluation, they're quite possible to design a unique benchmarks for evaluation. On the other hand, the customers are eager with smart phone because of its portability and fancy functions. Therefore, with the technology development, more advanced and powerful hardware will be integrated into the smart phone and large volume battery will effectively improve the performance of smart phone. The performance analysis of the smart phone has a bright future.

8. References

*The references are listed by the importance.

1. [Huang10]. Junxian Huang, Qiang Xu, Birjodh Tiwana, Z. Morley Mao, Ming Zhang, and Paramvir Bahl. 2010. "Anatomizing application performance differences on smartphones." In Proceedings of the 8th international conference on Mobile systems, applications, and services (MobiSys ‘10). ACM, New York, NY, USA, 165-178.http://doi.acm.org/10.1145/1814433.1814452.
2. [Keijzers08]. Jeroen Keijzers, Elke den Ouden, and Yuan Lu. 2008. "Usability benchmark study of commercially available smart phones: cell phone type platform, PDA type platform and PC type platform." In Proceedings of the 10th international conference on Human computer interaction with mobile devices and services (MobileHCI ‘08). ACM, New York, NY, USA, 265-272. http://doi.acm.org/10.1145/1409240.1409269.
3. [Agarwal07]. Yuvraj Agarwal, Ranveer Chandra, Alec Wolman, Paramvir Bahl, Kevin Chin, and Rajesh Gupta. 2007. "Wireless wakeups revisited: energy management for voip over wi-fi smartphones." In Proceedings of the 5th international conference on Mobile systems, applications and services (MobiSys ‘07). ACM, New York, NY, USA, 179-191. http://doi.acm.org/10.1145/1247660.1247682.
4. [MacKenzie02]. Scott MacKenzie. 2002. KSPC (Keystrokes per Character) as a Characteristic of Text Entry Techniques. In Proceedings of the 4th International Symposium on Mobile Human-Computer Interaction (Mobile HCI ‘02), Fabio Paternò (Ed.). Springer-Verlag, London, UK, 195-210.http://portal.acm.org/citation.cfm?id=666587.
5. [Silfverberg00]. Miika Silfverberg, I. Scott MacKenzie, and Panu Korhonen. 2000. Predicting text entry speed on mobile phones. In Proceedings of the SIGCHI conference on Human factors in computing systems (CHI ‘00). ACM, New York, NY, USA, 9-16. http://doi.acm.org/10.1145/332040.332044.
6. [Cuervo10]. Eduardo Cuervo, Aruna Balasubramanian, Dae-ki Cho, Alec Wolman, Stefan Saroiu, Ranveer Chandra, and Paramvir Bahl. 2010. "MAUI: making smartphones last longer with code offload." In Proceedings of the 8th international conference on Mobile systems, applications, and services (MobiSys ‘10). ACM, New York, NY, USA, 49-62. http://doi.acm.org/10.1145/1814433.1814441.
7. [IBM02]. IBM, 2002. Security in a Web Services world: A Proposed Architecture and Roadmap. IBM Developerworks. http://www.ibm.com/developerworks/library/specification/ws-secmap/
8. [LA06]. LA, 2006. The Liberty Alliance Project. http://www.projectliberty.org/.
9. [Castellucci09]. Steven J. Castellucci and I. Scott MacKenzie. 2009. "TnToolkit: a design and analysis tool for ambiguous, QWERTY, and on-screen keypads." In Proceedings of the 1st ACM SIGCHI symposium on Engineering interactive computing systems (EICS ‘09). ACM, New York, NY, USA, 55-60. http://doi.acm.org/10.1145/1570433.1570444.
10. [Paek10]. Jeongyeup Paek, Joongheon Kim, and Ramesh Govindan. 2010. "Energy-efficient rate-adaptive GPS-based positioning for smartphones." In Proceedings of the 8th international conference on Mobile systems, applications, and services (MobiSys ‘10). ACM, New York, NY, USA, 299-314. http://doi.acm.org/10.1145/1814433.1814463.
11. [Kerr09]. Simon Kerr, Hannah Thinyane, and Greg Foster. 2009. "Mobile phone performance analysis for camera based visual interactions." In Proceedings of the 2009 Annual Research Conference of the South African Institute of Computer Scientists and Information Technologists (SAICSIT ‘09). ACM, New York, NY, USA, 80-86. http://doi.acm.org/10.1145/1632149.1632161.
12. [Sani10]. Ardalan Amiri Sani, Lin Zhong, and Ashutosh Sabharwal. 2010." Directional antenna diversity for mobile devices: characterizations and solutions." In Proceedings of the sixteenth annual international conference on Mobile computing and networking (MobiCom ‘10). ACM, New York, NY, USA, 221-232.http://doi.acm.org/10.1145/1859995.1860021.
13. [Qian10]. Feng Qian, Zhaoguang Wang, Alexandre Gerber, Zhuoqing Morley Mao, Subhabrata Sen, and Oliver Spatscheck. 2010." Characterizing radio resource allocation for 3G networks." In Proceedings of the 10th annual conference on Internet measurement (IMC ‘10). ACM, New York, NY, USA, 137-150. http://doi.acm.org/10.1145/1879141.1879159.
14. [Walsh10]. Tanja Walsh, Piia Nurkka, and Rod Walsh. 2010." Cultural differences in smartphone user experience evaluation." In Proceedings of the 9th International Conference on Mobile and Ubiquitous Multimedia (MUM ‘10). ACM, New York, NY, USA, , Article 24 , 9 pages. http://doi.acm.org/10.1145/1899475.1899499.
15. [Falaki10]. Hossein Falaki, Ratul Mahajan, Srikanth Kandula, Dimitrios Lymberopoulos, Ramesh Govindan, and Deborah Estrin. 2010. "Diversity in smartphone usage." In Proceedings of the 8th international conference on Mobile systems, applications, and services (MobiSys ‘10). ACM, New York, NY, USA, 179-194. http://doi.acm.org/10.1145/1814433.1814453.
16. [Karlson10]. Amy K. Karlson, Shamsi T. Iqbal, Brian Meyers, Gonzalo Ramos, Kathy Lee, and John C. Tang. 2010. "Mobile taskflow in context: a screenshot study of smartphone usage." In Proceedings of the 28th international conference on Human factors in computing systems (CHI ‘10). ACM, New York, NY, USA, 2009-2018. http://doi.acm.org/10.1145/1753326.1753631.
17. [Srirama07] Satish Narayana Srirama, Matthias Jarke, Wolfgang Prinz. "A Performance Evaluation of Mobile Web Services Security", 3rd International Conference on Web Information Systems and Technologies (WEBIST 2007), pp. 386-392, March 3-6, 2007. http://arxiv.org/abs/1007.3644
18. [Riva08] Oriana Riva, Jaakko Kangasharju, "Challenges and Lessons in Developing Middleware on Smart Phones," Computer, pp. 23-31, October, 2008. http://ieeexplore.ieee.org/document/4640656/.
19. [Mattar07] Mattar, K., Sridharan, A., Zang, H., Matta, I., and Bestavros, A. "TCP over CDMA2000 Networks: A Cross-Layer Measurement Study." In Proceedings of PAM. 2007, 94-104. http://de.scientificcommons.org/43297186.
20. [Ghaderi07] Majid Ghaderi, Ashwin Sridharan, Hui Zang, Don Towsley, and Rene Cruz. "Modeling TCP in a multi-rate multi-user CDMA system," NETWORKING’07, Springer-Verlag, Berlin, Heidelberg, 2007, 403-414.http://portal.acm.org/citation.cfm?id=1772366.
21. [Chuah02] M. Chuah, W. Luo, and X. Zhang. "Impacts of Inactivity Timer Values on UMTS System Capacity," In Wireless Communications and Networking Conference, 2002. http://ieeexplore.ieee.org/document/993390.
22. [Liers07] Liers, F.; Burkhardt, C. and Mitschele-Thiel, A, "Static RRC Timeouts for Various Traffic Scenarios," Personal, Indoor and Mobile Radio Communications, 2007. PIMRC 2007. IEEE 18th International Symposium on , vol., no., pp.1-5, 3-7 Sept. 2007.http://ieeexplore.ieee.org/document/4394275/.
23. [Norklit01] O. Norklit, P. D. Teal, and R. G. Vaughan, "Measurement and evaluation of multi-antenna handsets in indoor mobile communication," in IEEE Transactions on Antennas and Propagation, vol. 49, issue 3, 2001.http://ieeexplore.ieee.org/document/918617.
24. [Holleis07] Holleis, P., Otto, F., Hussmann, H., and Schmidt, A., "Keystroke-level model for advanced mobile phone interaction," Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, New York: ACM (2007), 1505-1514.http://portal.acm.org/citation.cfm?id=1240624.1240851.
25. [MacKenzie07] MacKenzie, I. S. and Read, J. C., "Using paper mockups for evaluating soft keyboard layouts," Proceedings of CASCON 2007. Toronto: IBM Canada Ltd., (2007), 98-108.http://portal.acm.org/citation.cfm?id=1321222.
26. [Constandache09] Constandache, I.; Gaonkar, S.; Sayler, M.; Choudhury, R.R.; Cox, L.; , "EnLoc: Energy-Efficient Localization for Mobile Phones," INFOCOM 2009, IEEE , vol., no., pp.2716-2720, 19-25 April 2009.http://ieeexplore.ieee.org/document/5062218/.
27. [Lin10] K. Lin, A. Kansal, D. Lymberopoulos, and F. Zhao, "Energy-accuracy aware localization for mobile devices," In Proceedings of 8th International Conference on Mobile Systems, Applications, and Services (MobiSys’10), June 2010.http://research.microsoft.com/en-us/um/people/zhao/pubs/mobisys10aloc.pdf.
28. [Zhuang10] Zhenyun Zhuang, Kyu-Han Kim, and Jatinder Pal Singh. "Improving energy efficiency of location sensing on smartphones," In Proceedings of the 8th international conference on Mobile systems, applications, and services (MobiSys ‘10). ACM, New York, NY, USA, 2010, 315-330. http://doi.acm.org/10.1145/1814433.1814464.
29. [CanalysPress]. Canalys Press Release: R2009112.http://www.canalys.com/pr/2009/r2009112.html/.
30. [Speedtest]. Speedtest.net.http://www.speedtest.net/.
31. [FCCBroadbandtest]. FCC Comsumer Broadband Test.http://broadband.gov/qualitytest/.
32. [SPMark11] Rightware. SPMark. 2011. http://www.rightware.com/en/Benchmarking+Software.
33. [Gu03] Xiaohui Gu; Nahrstedt, K.; Messer, A.; Greenberg, I.; Milojicic, D.; , "Adaptive offloading inference for delivering applications in pervasive computing environments," Pervasive Computing and Communications, 2003. (PerCom 2003). Proceedings of the First IEEE International Conference on , vol., no., pp. 107- 114, 23-26 March 2003.http://ieeexplore.ieee.org/document/1192732/.
34. [Pung09] Hung Keng Pung; Tao Gu; Wenwei Xue; Palmes, P.P.; Jian Zhu; Wen Long Ng; Chee Weng Tang; Nguyen Hoang Chung; , "Context-aware middleware for pervasive elderly homecare", Selected Areas in Communications, IEEE Journal on , vol.27, no.4, pp.510-524, May 2009.http://ieeexplore.ieee.org/document/4909287/.
35. [Shye09] Alex Shye, Benjamin Scholbrock, and Gokhan Memik. 2009. "Into the wild: studying real user activity patterns to guide power optimizations for mobile architectures," In Proceedings of the 42nd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO 42). ACM, New York, NY, USA, 168-178.http://doi.acm.org/10.1145/1669112.1669135.
36. [Schumacher10] Schumacher, R, Ed. 2010. The Handbook of Global User Research. Morgan Kaufmann, Burlington, USA.http://www.elsevier.com/wps/find/bookdescription.cws_home/717479/description#description.
37. [Dearman08] Dearman, D. and Pierce, J. S. "It’s on my other computer!: computing with multiple devices," In Proc. CHI 2008, ACM Press (2008), 767-776.http://portal.acm.org/citation.cfm?id=1357177.
38. [Oulasvirta07] Oulasvirta, A. and Sumari, L. "Mobile kits and laptop trays: managing multiple devices in mobile information work," In Proc. CHI 2007, ACM Press (2007), 1127-1136.http://portal.acm.org/citation.cfm?id=1240795.
39. [Ra10] Moo-Ryong Ra, Jeongyeup Paek, Abhishek B. Sharma, Ramesh Govindan, Martin H. Krieger, and Michael J. Neely. 2010. "Energy-delay tradeoffs in smartphone applications," In Proceedings of the 8th international conference on Mobile systems, applications, and services (MobiSys ‘10). ACM, New York, NY, USA, 255-270.http://doi.acm.org/10.1145/1814433.1814459.

9. List of Acronyms