HOW TO: Make Udoo Neo an IoT Gateway of Sensors

Introduction
This HOW TO is the first of two. This one meant for beginners who just bought a Neo and want to connect it to the Internet and want to see their sensor data on the internet. The second will be about controlling the Neo Arduino in- and ouputs through a local website on your Neo so you can control it from everywhere in your house and from every connected device (smartphone, laptop, desktop) that has a web browser.

So this HOW TO is about sending sensor data to an IoT dashboard cloud supplier like Thingspeak. But there are more like for example data.sparkfun.com

Summary
1: Create a Thingspeak channel so we can upload data to this IoT cloud hosting.
2: Connect the Neo to the internet so it can connect to Thingspeak
3: Create a Python program on the Linux side to send the data to Thingspeak.
4: Create an Arduino Sketch for reading sensor data and sending it to the Linux side
5. Check your Thinspeak chanel and Confirm data transfer
6. Read multiple Sensors and send to Thingspeak

Let's begin! If you have any questions on this How To you are free to ask, just comment in this thread.

Step 1: Create a Thingspeak channel
Create Thingspeak channel with 1 temp field. You can use the following instruction: https://nl.mathworks.com/help/thingspeak/collect-data-in-a-new-channel.html

Step 2: Connect the Neo to the internet.
For connecting the Neo to the internet you can use the following instruction: Use Neo as a headless IoT Device

Step 3: Create a Python program on the Linux side to send the data to Thingspeak.
Copy this Python code, save it as UdooNeoSensor.py in your home folder on the Neo.
Replace the key value with your own Thingspeak Write API Key
Python is preinstalled on the Neo so you dont have to install anything for this.
After saving this python code you can run it ín a linux terminal console: sudo python UdooNeoSensor.py
We are using the Linux /dev/ttyMCC serial device that is implemented on the Neo for communicating between the Linux and Arduino side.
Tip: To copy the code open the Neo desktop through VNC Remote Desktop so you can copy/paste it from your web browser on your PC to the text editor (leafpad) on the Neo with the standard Ctrl-C Ctrl-V commands..

Code:

#!/usr/bin/env python
# This program logs a Arduino Sensor on an Udoo Neo to a Thingspeak Channel
# To use, get a Thingspeak.com account, set up a channel, and capture the Channel Key at https://thingspeak.com/docs/tutorials/
# Then paste your channel ID in the code for the value of "key" below.
# Then run as sudo python UdooNeoSensor.py

import httplib, urllib
import time
import serial
sleep = 16 # how many seconds to sleep between posts to the channel, API limit min 15 sec
key = 'Put your Thingspeak Channel Key here'  # Thingspeak channel to update

DEBUG = False # set to True to  get more debug prints in the Python console

#Create serial device for reading serial
arduino = serial.Serial('/dev/ttyMCC',115200,timeout=0)
arduino.flushOutput()
arduino.flushInput()
serial_buffer = ""
line = ""

temp_str = '0'
temp = 0

#init for arduino so Arduino knows it can send data.
arduino.write("I\n")

def ReadArduino(arduino_out):
    global serial_buffer, line
    arduino_out = ""
    serial_data = arduino.read()                 # Read 1 character
    while serial_data:                             # If there is a character proceed
        serial_buffer += serial_data            # Add character to buffer
        if DEBUG: print "ReadArduino:Serial buffer:",(repr(serial_buffer))
        # if there is a new line in it, then create Line string
        if "\r\n" in serial_buffer:
            if DEBUG: print "ReadArduino:New Line:",(repr(serial_buffer))
            line = serial_buffer
            serial_buffer = "" # empty Serial buffer
        serial_data = arduino.read()
        # if there is a Line string then finish reading the Line
        if line:
                # strip string for unwanted characters
                line = line.replace("\r\n", "")
                arduino_out = line
                if DEBUG: print "ReadArduino:arduino_out:", arduino_out
                line = ""
                if arduino_out == "": arduino_out = None
                # add the last character to the new buffer
                serial_buffer += serial_data
                return arduino_out

#Report Udoo Neo Arduino Sensor data to Thingspeak Channel
def writesensordata():
    global temp, temp_str
    while True:
        #read arduino data and make it a integer value
        temp_str = ReadArduino(temp_str)
        if temp_str is None:
            temp_str = '0'
        temp = int(temp_str)
        params = urllib.urlencode({'field1': temp, 'key':key })
        headers = {"Content-typZZe": "application/x-www-form-urlencoded","Accept": "text/plain"}
        conn = httplib.HTTPConnection("api.thingspeak.com:80")
        try:
            conn.request("POST", "/update", params, headers)
            response = conn.getresponse()
            print temp
            print response.status, response.reason
            data = response.read()
            conn.close()
        except:
            print "connection failed"
        break
#sleep for desired amount of time
if __name__ == "__main__":
        while True:
                writesensordata()
                time.sleep(sleep)

Step 4: Create Arduino Sketch for reading sensor end sending it to Serial
Use the following code to read an arduino analog (temperature) sensor on pin A0 and send it to serial (see also https://www.arduino.cc/en/Tutorial/AnalogReadSerial for more info). I modified the code a bit to slow down the read frequency and to be sure the data is send to the Linux side of the Neo.
Copy the code in a new sketch (as a start use the IDE on the Neo) and save it. Compile and upload it to the Arduino side of the Neo.
You can check the results of this sketch with the Serial Monitor of the Arduino IDE.
You first have to send a character to the Arduino in order to get the sensor data displayed (every 16 seconds). Be sure the Python program is not running else you will get a reading conflict of the /dev/ttytMCC serial device.

Code:

void setup() {
  // initialize serial communication:
  Serial.begin(115200);
  delay(100);
  pinMode(13, OUTPUT);
}

// the loop routine runs over and over again forever:
void loop() {
  while ( Serial.available() > 0 ) {
  // read the input on analog pin 0:
  int sensorValue = analogRead(A0);
  // print out the value you read:
  digitalWrite(13, HIGH);  // turn the LED on (HIGH is the voltage level)
  delay(1000);  // wait for a second.
  Serial.println(sensorValue); // If LED stays high longer you have serial timeout
  digitalWrite(13, LOW);  // turn the LED off by making the voltage LOW
  delay(10000);  // wait for a 10 seconds to prevent buffer overflow
  }
}

Step 5: Check your Thinspeak channel and Confirm data transfer
Be sure the python program and the arduino sketch are running.
Open your Thingspeak channel and you should see the temperature uploading into thinspeak website.

You can check my data here: https://thingspeak.com/channels/144274
I did not change the value of the sensor so this is a very straight line
You can download the values and there is an update every 16 seconds:
2016-08-10 23:22:00 UTC,12,685
2016-08-10 23:22:17 UTC,13,684
2016-08-10 23:22:33 UTC,14,685
2016-08-10 23:22:49 UTC,15,684
2016-08-10 23:23:06 UTC,16,684
2016-08-10 23:23:22 UTC,17,685

Step 6: Read multiple Sensors and send to Thingspeak
If you have more sensors you can send all data in one string (use Serial.print() and end with a Serial.println() ) to the A9 and split this string into multiple values again in Python. See an example in the comments. http://www.udoo.org/forum/index.php?posts/24362

Or you could use a sensor ID with the value.
The Arduino sketch and Python program in the How To: control Arduino from webpage is an example for using multiple sensors.

References
Use Neo as a headless IoT Device
UDOO NEO Serial Communication Examples
Python programming for beginners

Disclaimer: I am not a programmer, just a computer enthousiast. The code provide is tested but is not guaranteed to be the best in class. The code and How To is written for beginners to give a start in using the Neo. At the moment this How To is work in progress so updates are foreseen.
[/URL]

 

The second How to: Control Arduino from Web page is now online.

 

thanks for your instructions and guidelines waltervl !!!! i have developed a Light Sensor scenario which works perfectly with ThingSpeak!

 

Has anyone tried to update 2 or more data fields in the Thingspeak channel?

 

I did not but it should not be that difficult.
To write 2 values (ie temp and value2) change in the python script:
params = urllib.urlencode({'field1': temp, 'key':key })
to
params = urllib.urlencode({'field1': temp, 'field2': value2, 'key':key })

To get two or more values out of the arduino part of the Neo go to the other How to: Control Arduino from Web page

 

Thanks! It's cool.
How can I send date from Bricks directly?

 

It's my code

Code:

# A library (example) to get the current values for the outboard sensors such as
# Temperature sensor
# Barometer
import httplib, urllib
import time
import serial
sleep = 16 # how many seconds to sleep between posts to the channel, API limit min 15 sec
key = '99QLS10HUFWWMYIR'  # Thingspeak channel to update

from neo import Temp #import libraries
from neo import Barometer
from time import sleep # import for delays

temp = Temp() # init objects p.s. I auto initialize/reset the modules on these calls
baro = Barometer()
key = '99QLS1********'  # Thingspeak channel to update

try:
   conn = httplib.HTTPConnection("api.thingspeak.com:80")

except:
    print("Couldn't connect to the API, check your Internet connection")
    sys.exit(0)

#Report Udoo Neo Arduino Sensor data to Thingspeak Channel
def writesensordata():
    global temp, temp_str
while True:
        #read arduino data and make it a integer value
        temp_str = baro.getTemp("c") # slower than temp sensor but still works same as temp sensor replace c with f for different modes
        if temp_str is None:
            temp_str = '0'
        temp = int(temp_str)
        params = urllib.urlencode({'field1': temp, 'key':key })
        headers = {"Content-typZZe": "application/x-www-form-urlencoded","Accept": "text/plain"}
        conn.request("POST", "/update", params, headers)
        response = conn.getresponse()
        print temp
        print response.status, response.reason
        data = response.read()
        conn.close()
#print "" # newline
        sleep(5) # wait a (5) second

 

And what is not working? What errors do you get?

 

Everything is working
But I see echo: I/O error

 

Where is that I/O error coming from? Reading the sensors?
Are you able to print the sensor values to the console so to be sure you can read the brick sensors.
You could program more print statements so you can debug.

 

I can read sensors and send of the data on thingspeak. But first string i/o error. After this all ok.

Code:

# A library (example) to get the current values for the outboard sensors such as
# Temperature sensor
# Barometer
import httplib, urllib
import time
import serial
sleep = 16 # how many seconds to sleep between posts to the channel, API limit min 15 sec
key = '99QLS10HUFWWMYIR'  # Thingspeak channel to update

from neo import Temp #import libraries
from neo import Barometer
from time import sleep # import for delays

temp = Temp() # init objects p.s. I auto initialize/reset the modules on these calls
baro = Barometer()
key = '99QLS****'  # Thingspeak channel to update

try:
   conn = httplib.HTTPConnection("api.thingspeak.com:80")

except:
    print("Couldn't connect to the API, check your Internet connection")
    sys.exit(0)

#Report Udoo Neo Arduino Sensor data to Thingspeak Channel
def writesensordata():
    global temp, temp_str
while True:
        #read arduino data and make it a integer value
        temp_str = baro.getTemp("c") # slower than temp sensor but still works same as temp sensor replace c with f for different modes
        if temp_str is None:
            temp_str = '0'
        temp = int(temp_str)
        bar_str = baro.getPressure() # gets the pressure in kPA
        barHg=7.50063*bar_str
        if bar_str is None:
            bar_str = '0'
        bar = int( barHg)
        params = urllib.urlencode({'field1': temp, 'field2': bar, 'key':key })
        headers = {"Content-typZZe": "application/x-www-form-urlencoded","Accept": "text/plain"}
        conn.request("POST", "/update", params, headers)
        response = conn.getresponse()
        print temp
        print bar
        print response.status, response.reason
        data = response.read()
        conn.close()
#print "" # newline
        sleep(1) # wait a (5) second

 

Ok, then all OK.
I would not send data every second but do it every minute or so. I also have read somewhere that thingspeak only accepts data every 15 seconds.

 

Thanks.
How can I autoruning python script after starting of ubuntu.

 

You could start your script at the end of /etc/rc.local. 'Better' would be to create a Linux service.

 

This would be the instruction to do so from link:
Put this file in /etc/init
mystartupscript.conf

Code:

start on runlevel [2345]
stop on runlevel [!2345]

exec /path/to/script.py

By placing this conf file there you hook into ubuntu's upstart service that runs services on startup.
manual starting/stopping is done with "sudo service mystartupscript start" and "sudo service mystartupscript stop"

 

Does not work

/etc/init/mystartupscript.conf
------
start on runlevel [2345]
stop on runlevel [!2345]

exec /home/udooer/Desktop/udoo_thing_job.py

 

What does the logfile /var/log/upstart/[servicename] say?
And perhaps you have to make the .py file executable. or use

Code:

exec python /path/to/your/script.py

 

logfile

Code:

sh: echo: I/O error

Traceback (most recent call last):

  File "/home/udooer/Desktop/udoo_thing_job.py", line 42, in <module>

    conn.request("POST", "/update", params, headers)

  File "/usr/lib/python2.7/httplib.py", line 1017, in request

    self._send_request(method, url, body, headers)

  File "/usr/lib/python2.7/httplib.py", line 1051, in _send_request

    self.endheaders(body)

  File "/usr/lib/python2.7/httplib.py", line 1013, in endheaders

    self._send_output(message_body)

  File "/usr/lib/python2.7/httplib.py", line 864, in _send_output

    self.send(msg)

  File "/usr/lib/python2.7/httplib.py", line 826, in send

    self.connect()

  File "/usr/lib/python2.7/httplib.py", line 807, in connect

    self.timeout, self.source_address)

  File "/usr/lib/python2.7/socket.py", line 553, in create_connection

    for res in getaddrinfo(host, port, 0, SOCK_STREAM):

socket.gaierror: [Errno -2] Name or service not known

 

Looks like the same error as before where you said everything works. Does it really work? Do you see data coming in on your Thingspeak channel with the normal program?

 

After run udoo_thing_job.py
I see
Terminal:
sh: echo: I/O error
17
729
200 OK
.....
on ThingSpeak I see
https://thingspeak.com/channels/138029
-------
ThingSpeak script works from terminal but don't works from "sudo service mystartupscript start"
Ubidots script works. I run script through "sudo service mystartupscript start". But automatically don't start. I must run "sudo service mystartupscript start" through terminal.

Ubidots code:

Code:

#!/usr/bin/python
# A library (example) to get the current values for the outboard sensors such as
# Temperature sensor
# Barometer

from neo import Temp #import libraries
from neo import Barometer

from time import sleep # import for delays

temp = Temp() # init objects p.s. I auto initialize/reset the modules on these calls
baro = Barometer()

# Ubidots send
import time
import sys
import commands
import subprocess
from ubidots import ApiClient

try:

    api = ApiClient(token="UVVUNvgF76upGETU3v9QcHgLR0WMDQ")

except:

    print("Couldn't connect to the API, check your Internet connection")
    sys.exit(0)

while True:
    tempVal = temp.getTemp("f") # replace f with c to get celcius
    #print "Current temp from sensor 1: "+str(tempVal) #need to turn into string before building strings
       #Temperature sensor - tempVal
        api.save_collection([{'variable':'5777bbe67625427a308c0c91','value':tempVal}])

    pressureVal = baro.getPressure() # gets the pressure in kPA
        pressureValinHg=7.50063*pressureVal
    #print "Current pressure in (kPa):  "+str(pressureVal)
       #Barometer - pressureVal
        api.save_collection([{'variable':'5779289a76254218a67eacaf','value':pressureValinHg}])

    tempFromBaro = baro.getTemp("c") # slower than temp sensor but still works same as temp sensor replace c with f for different modes
    #print "Current temp from sensor 2: "+str(tempFromBaro)
      #TempFromBaro
        api.save_collection([{'variable':'5777bbee7625427a763c7332','value':tempFromBaro}])

    #print "" # newline
    sleep(1800) # wait a (5) second

----------------------------------------------------

ThingSpeak code:

Code:

#!/usr/bin/python
# A library (example) to get the current values for the outboard sensors such as
# Temperature sensor
# Barometer
import urllib
import httplib
import urlparse
import time

sleep = 16 # how many seconds to sleep between posts to the channel, API limit min 15 sec
key = '99QLS10HUFWWMYIR'  # Thingspeak channel to update

from neo import Temp #import libraries
from neo import Barometer
from time import sleep # import for delays

temp = Temp() # init objects p.s. I auto initialize/reset the modules on these calls
baro = Barometer()
key = '99QLS10HUFWWMYIR'  # Thingspeak channel to update

try:
   conn = httplib.HTTPConnection("api.thingspeak.com:80")

except:
    print("Couldn't connect to the API, check your Internet connection")
    sys.exit(0)

#Report Udoo Neo Arduino Sensor data to Thingspeak Channel
def writesensordata():
    global temp, temp_str
while True:
        #read arduino data and make it a integer value
        temp_str = baro.getTemp("c") # slower than temp sensor but still works same as temp sensor replace c with f for different modes
        if temp_str is None:
            temp_str = '0'
        temp = int(temp_str)
        bar_str = baro.getPressure() # gets the pressure in kPA
        barHg=7.50063*bar_str
        if bar_str is None:
            bar_str = '0'
        bar = int( barHg)
        params = urllib.urlencode({'field1': temp, 'field2': bar, 'key':key })
        headers = {"Content-typZZe": "application/x-www-form-urlencoded","Accept": "text/plain"}
        conn.request("POST", "/update", params, headers)
        response = conn.getresponse()
        print temp
        print bar
        print response.status, response.reason
        data = response.read()
        conn.close()
#print "" # newline
        sleep(1800) # wait a (5) second