CIS 694EEC 693 Android Sensor Programming Lecture 13

CIS 694/EEC 693 Android Sensor Programming Lecture 13 Wenbing Zhao Department of Electrical Engineering and Computer Science Cleveland State University wenbing@ieee. org 9/26/2020 CIS 470: Mobile App Development 1

Introduction to Sensor Programming n n n Understanding various sensor framework callbacks Using sensors in the foreground activity Listing the available sensors on a device Knowing individual sensors' capabilities Getting the sensor values and updating the user interface Monitoring sensor values in the background service 9/26/2020 Android Sensor Programming 2

Understanding the sensor framework callbacks n n n The two most important callbacks of the sensor framework are the on. Sensor. Changed() and on. Accuracy. Changed() methods In order to write efficient sensor code, it's important to understand when these methods are called, and what processing we can do in them These callbacks are methods of the Sensor. Event. Listnener interface, which needs to be implemented in the class where the callbacks are to be received: @Override public void on. Sensor. Changed(Sensor. Event event) { } @Override public void on. Accuracy. Changed(Sensor sensor, int accuracy) { } 9/26/2020 Android Sensor Programming 3

On on. Sensor. Changed() callback n n Depending on the type of reporting mode of the sensor, this method will be called, either at regular frequency (Continuous mode) or whenever there is a change in the value of the sensors from the previously reported value (On the change mode) The on. Sensor. Changed() method provides the sensor values inside the float value[] array of the Sensor. Event object q n These sensor values are different from the previously reported values There may be instances where the OS can choose to report sensor values at a different frequency than specified at the time of the registration of the listener q 9/26/2020 Generally, this happens because the OS is heavily loaded and busy with performing some other, more important, processing tasks Android Sensor Programming 4

On on. Accuracy. Changed() callback n n n Whenever the OS chooses to change (increase or decrease) the accuracy of the sensor values, it informs your listener using this callback In this method call, it sends the new accuracy value (Integer) and the sensor object for which the accuracy has changed Generally, this change in accuracy happens rarely q q 9/26/2020 Only when the OS goes into battery-saver mode or gets involved with important heavy-processing tasks When the OS gets busy, it reduces the accuracy, and when the OS becomes free, it increases the accuracy Android Sensor Programming 5

Major Steps in Sensor Programming 9/26/2020 Android Sensor Programming 6

Using sensors in the foreground activity n The first step is to implement our activity with the Sensor. Event. Listener interface so that our activity can receive Sensor. Event through the on. Sensor. Changed() method import android. app. Activity; import android. content. Context; import android. hardware. Sensor. Event; import android. hardware. Sensor. Event. Listener; import android. hardware. Sensor. Manager; import android. os. Bundle; public class Sensor. Activity extends Activity implements Sensor. Event. Listener{ 9/26/2020 Android Sensor Programming 7

Using sensors in the foreground activity n In step 2, we create the instance of Sensor. Manager from the system sensor service. Then, we create the Sensor object from Sensor. Manager private Sensor. Manager m. Sensor. Manager; private Sensor m. Sensor; @Override protected void on. Create(Bundle saved. Instance. State) { super. on. Create(saved. Instance. State); set. Content. View(R. layout. activity_main); m. Sensor. Manager = (Sensor. Manager)this. get. System. Service(Context. SENSOR_SERVICE ); if(m. Sensor. Manager. get. Default. Sensor (Sensor. TYPE_GYROSCOPE)!= null){ m. Sensor = m. Sensor. Manager. get. Default. Sensor(Sensor. TYPE_GYROSCOPE); } } 9/26/2020 Android Sensor Programming 8

Using sensors in the foreground activity n n In step 3, we register the listener for the callbacks. The registration of the listener is done using the register. Listener() method of Sensor. Manager, which accepts the objects of the Sensor. Event. Listener interface, the Sensor class, and the sampling period As a best practice, we should register the listener in the on. Resume() method, and unregister it in the on. Pause() method 9/26/2020 @Override protected void on. Resume() { super. on. Resume(); m. Sensor. Manager. register. Listener(this, m. Sensor, Sensor. Manager. SENSOR_DELAY_NORMA L); } @Override protected void on. Pause() { super. on. Pause(); m. Sensor. Manager. unregister. Listener(this); } @Override protected void on. Destroy() { super. on. Destroy(); m. Sensor. Manager = null; m. Sensor = null; } Android Sensor Programming 9

Using sensors in the foreground activity n In step 4, we define the callbacks on sensor events q q The on. Sensor. Changed() method is the main callback of the Senor. Event. Listener interface, which passes the sensor values in the form of the Sensor. Event object. The on. Accuracy. Changed() method is the second method that gets called whenever there is a change in the accuracy of the sensor values @Override public void on. Sensor. Changed(Sensor. Event event) { //event. values[] (do something with sensor values) //event. timestamp (do something with timestamp) } @Override public void on. Accuracy. Changed(Sensor sensor, int accuracy) { //Do something with changed accuracy //This method is mandatory to defined } 9/26/2020 Android Sensor Programming 10

n n n Listing the available sensors on a device There are multiple sensors available on a device. We will learn how to get a list of all the available sensors. We will be populating the names of the available sensors in a list and will be displaying it on the screen using List. View On top of this, the app will allow you to select a particular sensor to display the characteristics of the sensor Then, you may choose to open another activity to see the sensor values 9/26/2020 Android Sensor Programming 11

n n Listing the available sensors on a device Listing the sensors supported by the current device Class declaration public class Sensor. List. Activity extends Activity implements On. Item. Click. Listener { private Sensor. Manager m. Sensor. Manager; private List. View m. Sensor. List. View; private List. Adapter m. List. Adapter; private List<Sensor> m. Sensors. List; 9/26/2020 Android Sensor Programming 12

n Listing the available sensors on a device In the on. Create() method, we instantiate our Sensor. Manager, List. View, and List. Adaptor objects. We will also set the item click listener on our List. View and populate the sensor list using the get. Sensor. List() method of Sensor. Manager @Override protected void on. Create(Bundle saved. Instance. State) { super. on. Create(saved. Instance. State); set. Content. View(R. layout. activity_main); m. Sensor. Manager = (Sensor. Manager)this. get. System. Service (Context. SENSOR_SERVICE); m. Sensors. List = m. Sensor. Manager. get. Sensor. List(Sensor. TYPE_ALL); m. Sensor. List. View = (List. View)find. View. By. Id(R. id. session_list); m. List. Adapter = new List. Adapter(); m. Sensor. List. View. set. Adapter(m. List. Adapter); m. Sensor. List. View. set. On. Item. Click. Listener(this); } 9/26/2020 Android Sensor Programming 13

n Listing the available sensors on a device List. Adapter class for sensor listing (as part of the Sensor. List. Activity) private class List. Adapter extends Base. Adapter{ private Text. View m. Sensor. Name; @Override public int get. Count() { return m. Sensors. List. size(); } @Override public Object get. Item(int position) { return m. Sensors. List. get(position). get. Name(); } @Override public long get. Item. Id(int position) { return position; } @Override public View get. View(int position, View convert. View, View. Group parent) { if(convert. View==null){ convert. View = get. Layout. Inflater(). inflate(R. layout. list_rows, parent, false); } m. Sensor. Name = (Text. View)convert. View. find. View. By. Id(R. id. sensor_name); m. Sensor. Name. set. Text(m. Sensors. List. get(position). get. Name()); return convert. View; 9/26/2020 14 Android Sensor Programming }

n n Listing the available sensors on a device Implement on. Item. Click() listener for List. View. From this, we take the user to a new Sensor. Capability. Activity, which will show all the individual sensor details We pass the sensor type to the next activity so that it can identify the right sensor type. @Override public void on. Item. Click(Adapter. View<? > parent, View view, int position, long id) { Intent intent = new Intent(get. Application. Context(), Sensor. Capability. Activity. class); intent. put. Extra(get. Resources(). get. Resource. Name(R. string. sensor_type), m. Sensors. List. get(position). get. Type()); start. Activity(intent); } 9/26/2020 Android Sensor Programming 15

n Listing the available sensors on a device UI Layout for Sensor. List. Activity (activity_main. xml) <? xml version="1. 0" encoding="utf-8"? > <Linear. Layout xmlns: android="http: //schemas. android. com/apk/res/android" android: layout_width="match_parent" android: layout_height="match_parent" android: orientation="vertical" android: background="@android: color/white"> <List. View android: id="@+id/session_list" android: layout_width="fill_parent" android: layout_height="0 dip" android: layout_weight="1" android: cache. Color. Hint="@android: color/transparent" android: divider="@drawable/list_raw_divider_gradient" android: divider. Height=". 5 dip" /> </Linear. Layout> 9/26/2020 Android Sensor Programming 16

n Listing the available sensors on a device UI Layout for Sensor. List. Activity (list_rows. xml) <? xml version="1. 0" encoding="utf-8"? > <Linear. Layout xmlns: android="http: //schemas. android. com/apk/res/android" android: layout_width="fill_parent" android: layout_height="wrap_content" android: orientation="horizontal" > <Text. View android: id="@+id/sensor_name" style="@style/list_raw_text_style" /> </Linear. Layout> 9/26/2020 Android Sensor Programming 17

Knowing individual sensors' capabilities n n n Android phones are manufactured by different OEMs, which use different vendors to get their sensors. It is very much possible that two different Android phones have different gyroscope sensors, which will have different ranges and other properties We will create an activity to show the ranges and other properties for each sensor Show sensor properties in individual Text. View 9/26/2020 public class Sensor. Capability. Activity extends Activity { private Sensor. Manager m. Sensor. Manager; private int m. Sensor. Type; private Sensor m. Sensor; private Text. View m. Sensor. Name. Text. View; private Text. View m. Sensor. Maximum. Range. Text. View; private Text. View m. Sensor. Min. Delay. Text. View; private Text. View m. Sensor. Power. Text. View; private Text. View m. Sensor. Resolution. Text. View; private Text. View m. Sensor. Vendor. Text. View; private Text. View m. Sensor. Version. Text. View; Android Sensor Programming 18

Knowing individual sensors' capabilities n capability_layout. xml <Relative. Layout xmlns: android="http: //schemas. android. com/apk/res/android" xmlns: tools="http: //schemas. android. com/tools" android: layout_width="match_parent" android: layout_height="match_parent" android: background="@android: color/white"> <Text. View android: id="@+id/sensorname" android: layout_width="wrap_content" android: layout_height="wrap_content" android: text="Sensor Name" android: text. Color="@android: color/black" android: text. Style="bold" android: layout_margin. Top="10 dp" android: layout_margin. Left="10 dp" android: text. Size="18 sp" /> <Text. View android: id="@+id/sensor_name" android: layout_width="wrap_content" android: layout_height="wrap_content" android: text. Color="@android: color/darker_gray" android: layout_below="@+id/sensorname" android: layout_margin. Top="5 dp" android: layout_margin. Left="10 dp" android: text. Size="16 sp" /> 9/26/2020 Android Sensor Programming 19

Knowing individual sensors' capabilities n In the On. Created() method, we instantiate the Text. View, Sensor, and Sensor. Manager objects. We receive the sensor type through the get. Int. Extra() API of intent from the previous Sensor. List. Activity @Override protected void on. Create(Bundle saved. Instance. State) { super. on. Create(saved. Instance. State); set. Content. View(R. layout. capability_layout); Intent intent = get. Intent(); m. Sensor. Type = intent. get. Int. Extra(get. Resources(). get. Resource. Name(R. string. sensor_type), 0); m. Sensor. Manager = (Sensor. Manager)this. get. System. Service(Context. SENSOR_SERVICE); m. Sensor = m. Sensor. Manager. get. Default. Sensor(m. Sensor. Type); m. Sensor. Name. Text. View = (Text. View)find. View. By. Id(R. id. sensor_name); m. Sensor. Maximum. Range. Text. View = (Text. View)find. View. By. Id(R. id. sensor_range); m. Sensor. Min. Delay. Text. View = (Text. View)find. View. By. Id(R. id. sensor_mindelay); m. Sensor. Power. Text. View = (Text. View)find. View. By. Id(R. id. sensor_power); m. Sensor. Resolution. Text. View = (Text. View)find. View. By. Id(R. id. sensor_resolution); m. Sensor. Vendor. Text. View = (Text. View)find. View. By. Id(R. id. sensor_vendor); m. Sensor. Version. Text. View = (Text. View)find. View. By. Id(R. id. sensor_version); m. Sensor. Name. Text. View. set. Text(m. Sensor. get. Name()); m. Sensor. Maximum. Range. Text. View. set. Text(String. value. Of(m. Sensor. get. Maximum. Range()) ); m. Sensor. Min. Delay. Text. View. set. Text(String. value. Of(m. Sensor. get. Min. Delay())); m. Sensor. Power. Text. View. set. Text(String. value. Of(m. Sensor. get. Power())); m. Sensor. Resolution. Text. View. set. Text(String. value. Of(m. Sensor. get. Resolution())); m. Sensor. Vendor. Text. View. set. Text(String. value. Of(m. Sensor. get. Vendor())); 9/26/2020 m. Sensor. Version. Text. View. set. Text(String. value. Of(m. Sensor. get. Version())); Android Sensor Programming } 20

Knowing individual sensors' capabilities n n After knowing all the sensor properties, we would like to play with the sensor values via the on. Click. Sensor. Values() method. This method is attached to the button on the screen using the XML on. Click tag. We pass the sensor type to the next activity so that it can identify the right sensor type public void on. Click. Sensor. Values(View v) { Intent intent = new Intent(get. Application. Context(), Sensor. Values. Activity. class); intent. put. Extra(get. Resources(). get. Resource. Name(R. string. sen sor_type), m. Sensor. Type); start. Activity(intent); } 9/26/2020 Android Sensor Programming 21

Getting the sensor values and updating the user interface n n n After knowing all the sensor properties, we would like to play with the sensor values via the on. Click. Sensor. Values() method. This method is attached to the button on the screen using the XML on. Click tag. We pass the sensor type to the next activity so that it can identify the right sensor type We create a layout for this activity to fit a vast number of values for all sensor types q q q Sensors can have varied values such as temperature or pressure; a light and proximity sensor may have only one value sensors such as magnetometer, accelerometer, gyroscope, linear acceleration, and gravity have three values in the x, y, and z axes There are other sensors that can have more than three values, for example, rotational vector, geomagnetic rotational vector, game rotational vector, and un-calibrated gyroscope All the sensor values are passed in an array called values[], which is part of the Sensor. Event object 9/26/2020 Android Sensor Programming 22

Getting the sensor values and updating the user interface <Relative. Layout xmlns: android="http: //schemas. android. com/apk/res/android" xmlns: tools="http: //schemas. android. com/tools" android: layout_width="match_parent" android: layout_height="match_parent" android: background="@android: color/white" android: id="@+id/layout"> <Text. View android: id="@+id/sensorname" android: layout_width="wrap_content" android: layout_height="wrap_content" android: text="Sensor Values" android: text. Color="@android: color/black" android: text. Style="bold" android: layout_margin. Top="10 dp" android: layout_margin. Left="10 dp" android: text. Size="18 sp"/> <Text. View android: id="@+id/event 0" android: layout_width="wrap_content" android: layout_height="wrap_content" android: text. Color="@android: color/darker_gray" android: layout_below="@+id/sensorname" android: layout_margin. Top="5 dp" android: layout_margin. Left="10 dp" android: text. Size="16 sp"/> <Text. View android: id="@+id/event 1" android: layout_width="wrap_content" android: layout_height="wrap_content" android: text. Color="@android: color/darker_gray" android: layout_below="@+id/event 0" android: layout_margin. Top="5 dp" android: layout_margin. Left="10 dp" /> <Text. View android: id="@+id/event 2" android: layout_width="wrap_content" android: layout_height="wrap_content" android: text. Color="@android: color/darker_gray" android: layout_below="@+id/event 1" android: layout_margin. Top="5 dp" android: layout_margin. Left="10 dp" android: text. Size="16 sp" /> Incomplete, remaining similar 9/26/2020 Android Sensor Programming 23

Getting the sensor values and updating the user interface public class Sensor. Values. Activity extends Activity implements Sensor. Event. Listener{ private Sensor. Manager m. Sensor. Manager; private int m. Sensor. Type; private Sensor m. Sensor; private Text. View m. Event. Value_0; private Text. View m. Event. Value_1; private Text. View m. Event. Value_2; private Text. View m. Event. Value_3; private Text. View m. Event. Value_4; private Text. View m. Event. Value_5; private Text. View m. Event. Value_6; private Text. View m. Time; private Text. View m. Accuracy; 9/26/2020 @Override protected void on. Create(Bundle saved. Instance. State) { super. on. Create(saved. Instance. State); set. Content. View(R. layout. values_layout); Intent intent = get. Intent(); m. Sensor. Type =intent. get. Int. Extra(get. Resources(). get. Resource. Name (R. string. sensor_type), 0); m. Sensor. Manager = (Sensor. Manager)this. get. System. Service (Context. SENSOR_SERVICE); m. Sensor = m. Sensor. Manager. get. Default. Sensor(m. Sensor. Type; m. Event. Value_0 = (Text. View)find. View. By. Id(R. id. event 0); m. Event. Value_1 = (Text. View)find. View. By. Id(R. id. event 1); m. Event. Value_2 = (Text. View)find. View. By. Id(R. id. event 2); m. Event. Value_3 = (Text. View)find. View. By. Id(R. id. event 3); m. Event. Value_4 = (Text. View)find. View. By. Id(R. id. event 4); m. Event. Value_5 = (Text. View)find. View. By. Id(R. id. event 5); m. Event. Value_6 = (Text. View)find. View. By. Id(R. id. event 6); m. Time = (Text. View)find. View. By. Id(R. id. time); m. Accuracy = (Text. View)find. View. By. Id(R. id. accuracy); } Android Sensor Programming 24

Getting the sensor values and updating the user interface n As a best practice, we register and unregister the listener for the sensors in the on. Resume() and on. Pause() methods, so that we only get the values when the activity is in the foreground @Override protected void on. Resume() { super. on. Resume(); m. Sensor. Manager. register. Listener(this, m. Sensor, Sensor. Manager. SENSOR_DELAY_NORMAL); } @Override protected void on. Pause() { super. on. Pause(); m. Sensor. Manager. unregister. Listener(this); } 9/26/2020 Android Sensor Programming 25

Getting the sensor values and updating the user interface n n The same on. Sensor. Change() method is used to display the sensor values coming from different types of sensor The length of the values[] array is used to determine the number of values for that sensor, and it is also used in setting these sensor values into the same number of Text. View @Override public void on. Sensor. Changed(Sensor. Event event) { We also display the time and m. Event. Value_0. set. Text(String. value. Of(event. values[0])); accuracy given by the Sensor. Event m. Accuracy. set. Text(String. value. Of(event. accuracy)); m. Time. set. Text(String. value. Of(event. timestamp)); object on the screen if(event. values. length>1) { m. Event. Value_1. set. Text(String. value. Of(event. values[1])); We set the first value[0] without } if(event. values. length>2) { m. Event. Value_2. set. Text(String. value. Of(event. values[2])); checking, because we are sure that } if(event. values. length>3) { m. Event. Value_3. set. Text(String. value. Of(event. values[3])); the Sensor. Event object value's } if(event. values. length>4) { array will have at least one value m. Event. Value_4. set. Text(String. value. Of(event. values[4])); } if(event. values. length>5) { m. Event. Value_5. set. Text(String. value. Of(event. values[5])); } if(event. values. length>6) { m. Event. Value_6. set. Text(String. value. Of(event. values[6])); } } 9/26/2020 Android Sensor Programming 26

Processing the sensor values in the background service n n There will be cases where your app needs to listen and process the sensor values in the background: processing sensor values in service Example: detecting when someone picks up or handles the phone, then playing a small mp 3 sound q q Continuously process the gyroscope sensor values to detect this event The gyroscope gives the rate of rotation of the phone in each x, y, and z axis When your phone is kept still, there is no, or a very low rate of rotation reported as a value by gyroscope, but when the phone is picked up or handled, the value of rate of rotation goes very high Compute the magnitude of the rotation vector, if it exceeds a threshold (2. 5 is used), then we declare the detection of the event 9/26/2020 Android Sensor Programming 27

n Processing the sensor values in the background service: phone handling detection Create the service and initialize the required variables. The service implements the Sensor. Event. Listener interface so that it can receive the sensor callbacks public class Sensor. Service extends Service implements Sensor. Event. Listener { private Sensor. Manager m. Sensor. Manager; private Sensor m. Sensor; private Media. Player m. Media. Player; private boolean is. Playing = false; private float m. Threshold = 2. 5 f; 9/26/2020 Android Sensor Programming 28

Processing the sensor values in the background service n n n We will be using the service on. Create() method to initialize our variables Sensor. Manager and the Sensor object are initialized in the conventional way, as in the activity We also initialized the Media. Player object with a small MP 3 sound, which will be played when the phone-handling event triggers We use a combination of On. Completion. Listener on Media. Player and a Boolean is. Playing to make sure that a new instance of MP 3 is not played until the previous instance of MP 3 has finished playing Find and place an MP 3 sound file (mario. mp 3) in the raw folder created inside the res folder of the project structure 9/26/2020 Android Sensor Programming 29

Processing the sensor values in the background service @Override public void on. Create() { super. on. Create(); m. Sensor. Manager = (Sensor. Manager)this. get. System. Service(Context. SENSOR_SERVICE); m. Sensor = m. Sensor. Manager. get. Default. Sensor(Sensor. TYPE_GYROSCOPE); m. Media. Player = Media. Player. create(get. Application. Context(), R. raw. mario); m. Media. Player. set. On. Completion. Listener(new On. Completion. Listener(){ @Override public void on. Completion(Media. Player mp) { is. Playing = false; } }); } 9/26/2020 Android Sensor Programming 30

Processing the sensor values in the background service n Registering and unregistering listeners is done at different places in the service when compared with activity. In the service, we register the listener in the on. Start. Command() method, which is the time when the OS actually starts the service in the background. Similarly, we unregister the listener in the on. Destroy() method when the OS kills the service @Override public int on. Start. Command(Intent intent, int flags, int start. Id) { m. Sensor. Manager. register. Listener(this, m. Sensor, Sensor. Manager. SENSOR_DELAY_UI); return Service. START_NOT_STICKY; } @Override public void on. Destroy() { super. on. Destroy(); m. Sensor. Manager. unregister. Listener(this); } 9/26/2020 Android Sensor Programming 31

Processing the sensor values in the background service n n The core logic of the phone-handling algorithm is written in the on. Sensor. Changed() method. For each sensor value of the gyroscope, we apply this basic algorithm On detecting the phone handling event, we only play the MP 3 if it's not already being played. We do this extra check by maintaining the state of the is. Playing Boolean variable @Override public void on. Sensor. Changed(Sensor. Event event) { double rate. Of. Rotation = Math. sqrt(Math. pow(event. values[0], 2) + Math. pow(event. values[1], 2) + Math. pow(event. values[2], 2)); if(rate. Of. Rotation > m. Threshold) { if(!is. Playing){ m. Media. Player. start(); is. Playing = true; } } } 9/26/2020 Android Sensor Programming is. Playing will be set to false when the playing is over 32

Homework# 17 n Build the sensor listing app using the code provided 9/26/2020 CIS 470: Mobile App Development 33

Homework# 18 n Build an application with a service that continuous monitor phone handling activities and plays an mp 3 file on detection of such an activity 9/26/2020 CIS 470: Mobile App Development 34