A Smart Assistant for a Cerebral Palsy Patient

Pre-Requisites

Concept of power, ground, connectivity of a microcontroller, usage of breadboard

Introduction

Leading a life full of luxury is what a person with no disabilities can look up to, but when tragedy strikes, even leading a normal lifestyle is leading a luxurious life. Here, I provide you a solution, to provide a luxurious lifestyle for a person suffering from Cerebral Palsy. This is the Smart Assistant.

Problem Statement

More than 1 million people suffer from Cerebral Palsy each year in India. A person suffering from Cerebral Palsy experiences various difficulties in their day to day life, ranging from restricted muscle movements to frequent loss of memory. Our product aims to make life better for the people facing these issues.

Outlining the Solution

Since there’s restricted muscle movement and an occasional case of learning disability, the most obvious solution that comes to mind is deploying a smart device or system which involves ease of movement while providing full control to the said person, along with something to keep note of, rather than forgetting over and over again. Lastly monitoring the vitals will be essential since tremors are common so we need to accordingly send an alert to the designated personnel.

Implementation

Now, to start with our system, we have to choose a microcontroller. This can be either an Arduino Uno or Raspberry Pi or a Microbit. Our use case can be covered by all of these three but we want something more advanced than an Arduino and something less expensive than a Raspberry, so we go for Microbit. How Micro:bit takes an edge over Arduino can be understood through this comparison table.

The solution

To reduce the amount of effort required for movement, we use a gesture sensor which is inbuilt in our Micro:bit. This adds to its list of advantages because it reduces the cost of purchase and compatibility complexity when compared to an Arduino UNO.

We code this in Javascript and the output can be seen on our Micro:bit. To make the GUI more user friendly, we can pass on outputs which represent the input and the user can see whether their input was correctly registered or not. This can be easily achieved using the 5*5 LED Dot Matrix which will simply display the arrow for direction.

We’re using two Microbits, one to take the gesture as an input, and another to trigger the movement on the wheel.

The code for identifying the gesture on the first Microbit is given below :

 

		grove.onGesture(GroveGesture.Right, function () {
		    basic.showString("R")
		    x = 2
		    radio.sendNumber(x)
		})
		grove.onGesture(GroveGesture.Left, function () {
		    basic.showString("L")
		    x = 4
		    radio.sendNumber(x)
		})
		grove.onGesture(GroveGesture.Up, function () {
		    basic.showString("U")
		    x = 1
		    radio.sendNumber(x)
		})
		grove.onGesture(GroveGesture.Down, function () {
		    x = 3
		    basic.showString("D")
		    radio.sendNumber(x)
		})
		let x = 0
		basic.showIcon(IconNames.Chessboard)
		radio.setGroup(8)
		basic.forever(function () {
		    if (grove.measureInCentimeters(DigitalPin.P0) < 5) {
		        x = 3
		        radio.sendNumber(x)
		        basic.showIcon(IconNames.Sad)
		    }
		    if (grove.measureInCentimeters(DigitalPin.P1) < 5) {
		        x = 3
		        radio.sendNumber(x)
		        basic.showIcon(IconNames.No)
		    }
		    if (grove.measureInCentimeters(DigitalPin.P2) < 5) {
		        x = 3
		        radio.sendNumber(x)
		        basic.showIcon(IconNames.Heart)
		    }
		})
	

 

The important things to understand in this code snippets are:

 

		grove.onGesture
		grove.onGesture(GroveGesture.Right, function () {
		    basic.showString("R")
		    x = 2
		radio.sendNumber(x)
		}

	

 

- This signifies what action is supposed to be taken when a particular condition is matched, in this case, when a particular gesture is matched.

Here, when we give the “right gesture”, the string “R” is displayed on the Microbit and the number x (=2) is sent across the radio channel.

This same explanation can be interpreted for the subsequent lines of codes.

 

		basic.forever(function () {
		    if (grove.measureInCentimeters(DigitalPin.P0) < 5) {
		        x = 3
		        radio.sendNumber(x)
		        basic.showIcon(IconNames.Sad)
		    
        }
	

 

This and its subsequent lines of code mean that if the distance is less than 5 centimeters on any of the ultrasonic sensors, then the wheel chair should be stopped, in order to avoid the obstacle. Since this was a prototype, the distance used is 5 cms otherwise it is recommended to be at least half a metre away.

We can position the ultrasonic sensors on all four sides of the wheel chair.

To make sure that the impact of sudden brakes is reduced, a lower acceleration pattern is to be followed.

radio.setGroup() - This signifies what group this particular communication falls under. So in our case, the communication will be completed if the receiver code has the same group number ( here, 8 ).

What we need next is to trigger the motion of our wheels in such a manner that the chair turns according to the input. The code for this is given below :

 

		radio.onReceivedNumber(function (receivedNumber) {
		    if (receivedNumber == 1) {
		        basic.showNumber(1)
		        pins.digitalWritePin(DigitalPin.P13, 0)
		        pins.digitalWritePin(DigitalPin.P14, 0)
		        pins.digitalWritePin(DigitalPin.P11, 1)
		        pins.digitalWritePin(DigitalPin.P1, 1)
		        pins.analogWritePin(AnalogPin.P4, 1023)
		        pins.analogWritePin(AnalogPin.P5, 1023)
		    } else if (receivedNumber == 2) {
		        basic.showNumber(2)
		        pins.digitalWritePin(DigitalPin.P11, 0)
		        pins.digitalWritePin(DigitalPin.P1, 1)
		        pins.digitalWritePin(DigitalPin.P13, 0)
		        pins.digitalWritePin(DigitalPin.P14, 0)
		        pins.analogWritePin(AnalogPin.P4, 1023)
		        pins.analogWritePin(AnalogPin.P5, 1023)
		    } else if (receivedNumber == 3) {
		        basic.showNumber(3)
		        pins.digitalWritePin(DigitalPin.P1, 0)
		        pins.digitalWritePin(DigitalPin.P11, 0)
		        pins.digitalWritePin(DigitalPin.P13, 0)
		        pins.digitalWritePin(DigitalPin.P14, 0)
		        pins.analogWritePin(AnalogPin.P4, 1023)
		        pins.analogWritePin(AnalogPin.P5, 1023)
		    } else if (receivedNumber == 4) {
		        basic.showNumber(4)
		        pins.digitalWritePin(DigitalPin.P1, 0)
		        pins.digitalWritePin(DigitalPin.P11, 1)
		        pins.digitalWritePin(DigitalPin.P13, 0)
		        pins.digitalWritePin(DigitalPin.P14, 0)
		        pins.analogWritePin(AnalogPin.P4, 1023)
		        pins.analogWritePin(AnalogPin.P5, 1023)
		    } else {
		        basic.showNumber(0)
		    }
		})
		radio.setGroup(8)
	

 

The main things to understand for this code snippet are :

 

		radio.onReceivedNumber(function (receivedNumber) {
		    if (receivedNumber == 1) {
		        basic.showNumber(1)
		        pins.digitalWritePin(DigitalPin.P13, 0)
		        pins.digitalWritePin(DigitalPin.P14, 0)
		        pins.digitalWritePin(DigitalPin.P11, 1)
		        pins.digitalWritePin(DigitalPin.P1, 1)
		        pins.analogWritePin(AnalogPin.P4, 1023)
		        pins.analogWritePin(AnalogPin.P5, 1023)
		}

	

 

- This is the number you’re receiving from the previous code snippet via the inbuilt Bluetooth communication. We’ve pulled this off using the “radio.setGroup(8)” at the end in this code snippet. Every code snippet with this radio group number will be part of the communication.

If the received number is 1, which in our case signifies acceleration forward, then we’re powering the Digital Pins P1 and P11. This is because we want our chair to move forward. The diagram for different movement and pin mapping is shown below.

If 1,

P1 signifies forward movement of right wheel.

P11 signifies forward movement of left wheel.

P1 P11 signifies forward movement on both the wheels.

P13 P14 signifies reverse movement.

If 0, then the movement is restricted for a given pin.

This part is completely dependent on how you’ve configured your motor drivers. Accordingly, right, left and reverse movements have been defined.

The analog pins define the speed at which the wheelchair is supposed to move. Although we have obstacle prevention with us, this is important because even sudden brakes are not a comfortable experience.

So we can define our own speed prior to testing. This will be more user-specific and might vary.

Vitals

Since this can be implemented with just an Arduino Uno, we include the relevant vital measurement sensors to make the system efficient with one microcontroller.

Now, the two vitals that we’re considering for this project are heartbeat and body temperature. The codes have been segregated for clarity but in execution, it’s a single snippet of code.

Temperature :

 

		float val;
		int tempPin=A1;
		void setup() {
		  Serial.begin(9600);
		  pinMode(tempPin,INPUT);
		  }

		void loop() {
		val=analogRead(tempPin);
		float mv=(val/1024.0)*5000;
		float cel=mv/10;
		float farh=(cel)*9/5 + 32;
		Serial.print("Temperature=");
		Serial.print(cel);
		Serial.print("*C");
		Serial.println();
		delay(1000);
		Serial.print("Temperature=");
		Serial.print(farh);
		Serial.print("*F");
		Serial.println();  

		}
	

 

You can skip the understanding for this since codes can directly be copied and run on the Arduino IDE. The moderation is more requirement based.

In this code, like any other Arduino code, there are two functions, setup() and loop(). In setup(), we define the input pin mode, which in our case is an INPUT since we’re taking the input from the sensor attached to the tempPin ( A1 ).

In loop(), as the name suggests, we write the code which we want to be continuously executed repeatedly.

The essence is that the sensor is throwing a meaningful value, but not the exact temperature. Certain calculations are required to reach to that temperature, which have been illustrated above.

Heartbeat :

 

	int sensorPin = 0;
	double alpha = 0.75;
	int period = 100;
	double change = 0.0;
	double minval = 0.0;
	void setup ()
	{
	  Serial.begin (9600);
	}
	void loop ()
	{
	    static double oldValue = 0;
	    static double oldChange = 0;
	 
	    int rawValue = analogRead (sensorPin);
	    double value = alpha * oldValue + (1 - alpha) * rawValue;
	 
	    Serial.print (rawValue);
	    Serial.print (",");
	    Serial.println (value);
	    oldValue = value;
	 
	    delay (period);
	}
	
	

 

Same justification follows for this as well.

Sending data to cloud :

Here comes the important part. The code above will just compute the vital values for us. But we want them to be constantly monitored and an alert must be sent if a threshold is disturbed. So we go for a cloud-based approach.

In this case, we use ThingSpeak and NodeMCU, which is a WiFi module. Relevant libraries need to be included for this.

 

		#include ;
		#include ;
		#include ;
		#include 

		const char* ssid = "PVT";
		const int pResistor = A0; 
		const char* password = "PVT"; 

		WiFiClient client;

		unsigned long myChannelNumber = PVT; 
		const char * myWriteAPIKey = "PVT";

		void setup() 

		{ 

		Serial.begin(9600);  
		 
		pinMode(pResistor,INPUT);

		WiFi.begin(ssid, password);
		ThingSpeak.begin(client);

		}


		int count=0;
		  
		void loop() 

		  {
		 
		  int value = analogRead(pResistor);
		  if((value<20)|(value>50))
		  
		  {
		  
		    count+=1;
		    Serial.println(count);
		  
		  }
		  
		  Serial.println(value);
		  
		  ThingSpeak.writeField(myChannelNumber,4,count,myWriteAPIKey);
		  

		}
	

 

The reason we use a photoresistor here is because we can just use an LED reflecting light against the photoresistor, situated towards the ends of both the handles of the armrest on the wheelchair.

What this does is, if the person falls by any chance, he’ll obstruct this constant LED light falling on the photoresistor, hence causing a drastic change in the output of the photoresistor, implying that the person has fallen. An alert can be sent to nearby family members.

Now, ThingSpeak is basically a cloud platform which is used to represent, analyse, react to, and store data. We can simply create a channel of our own by going to the Thingspeak website.

Firstly, create an account.

Sign in.

Go to channels

Then, create a channel.

Give any name you want, and name the fields based on the number of requirements. In our case, 3; heartbeat, temperature and the value of the photoresistor. You can name them as you wish.

You can ignore the rest of the fields and proceed to creating your channel.

You can go through various settings under the channel settings tab and make changes as per your convenience but here are some basic things you need to be aware about.

Under the channel settings tab, you’ll see a screen similar to this.

In our code, just above the setup() function definition, you can see two variables, myChannelNumber and myWriteAPIKey. This channel number that you see on your screen is what you’re supposed to copy into myChannelNumber. And, what you’ll retrieve below is what you’re supposed to copy into myWriteAPIKey.

Head over to the API Keys section. Here you have access to the keys that’ll make the connection possible between your cloud and your code.

Now you can do two things here, read and write. Write, here means writing/ publishing your retrieved values from sensors onto your channel on Thingspeak. Read can be used to read data from channels. Our use case is limited to Write here.

So you copy this writekey to the appropriate variable as discussed above and head over to the Private View tab if you’ve chosen to not share your channel with anyone, or the Public View tab if you have.

Now, the last thing you need to do is connect your WiFi Module NodeMCU to a nearby internet connection. You probably already guessed it, that’s what the variables ssid and password are there for in your code.

If you have a hotspot/ WiFi connection named “Hello World” and password is “C#3.14” then ssid will be initialized as the former and password as the later. Turn on your router/ hotspot and execute the code. You’ll start seeing output on your screen.

Here, you can see an example output.

Next thing is acting upon this data, in case a threshold has been breached. We do that, and a lot more using the Apps that Thingspeak provides us with. You can discover various Apps as per your requirement, we’re gonna look at Reacts for now.

So, we make a react named “React 1” which is executed if the “String” ( Condition Type ) in the field 5 ( which signifies smoke for my Smart Farm project ) is not equal to 0 which implies that there’s fire in the farm. The string “FIRE, FIRE!” will be tweeted from the Twitter account you choose to connect in the subsequent field.

This is how our React looks like.

Once the condition is breached, this is how the react performs its execution.

Similarly, you can make your own reactions for your use case. In our smart assistant system, the react will work every time the Photoresistor’s value isn’t what it should be when an LED is reflecting against its surface.

The last and final thing you need in this project is a voice memo, so that it can store whatever the person has to say or remember for later. For this you can simply refer to this paper A Flexible Speech Recognition System for Cerebral Palsy Disabled and implement the execution on your own.

This can be an added feature for your product, which isn’t there with the base model since not everyone suffers from loss of memory.

Future Scope

The future scope here mainly includes more features which take care of specific use cases which increases the cost of our product, such as a manual robotic arm which wipes the saliva off a person’s face, because uncontrollable drooling is a problem for some people due to a non functioning lower jaw. You can mount a camera and judge if the face needs cleaning, and the arm will do the rest. Or there can be a trigger gesture specially crafted for this.

ABOUT THE AUTHOR