Data logging is very important in Solar PV Application. Normally most of the Solar PV System (on-grid) and high end (Off-Grid) provides app or software for real time and data logging function by the inverter’s manufacturer so that user can trace back the generation pattern of the system and also user can monitor the real time and condition of the Solar PV generation anywhere anytime. It is well known as Internet of Things (IOT). The inverter’s manufacturer will has its own web server (cloud) to store the incoming data of all the registered clients. Each inverter will send data to its cloud via a unique channel. Client will be granted a unique channel and there is no issue to monitor more than 10 inverters at the same time as all inverters can be subscribed to 1 channel. With the access of identical and unique channel, all devices can be interconnect and communicate to each other and become a smart electrical system.
Imagine if you have a smartphone acts as a portable display to monitor and read values or sometimes you can control the other devices which are scattered all around the world while all the related devices can also be talking and control each other automatically. Does it sounds interesting to you?
For me, Internet of Things (IOT) can be divided into 2 main categories. The first category is action or execution application which we use internet to link all the device and make action such as turn ON/OFF or dim brightness, provide alarm or signals etc. It requires physical action towards devices. Smart Home system is one of the application of this kind. For more info about IOT execution application, kindly go to my other blog here. The second category is monitoring and data acquisition application which we use internet to collect values from devices or from websites and store on a web server. Meteorological Weather station is one of the example of this application. In this blog page, we focus on this application.
At first, we need to get a web server from an internet broker that could provides us a data analytic function which could connect our devices and stores all the data on cloud. The broker will provide us a unique main channel or network for all our devices so that only our-self can access using the unique key. In this project we will be using ThingSpeak as our internet broker for analytic application such as collects our sensor module data for real time monitoring and as a cloud storage space so that we could download the data for analysis if required. I choose ThinkSpeak because it is FREE !! Of cause once you have mastered the skills and knowledge, you may consider upgrade to more professional paid plan. As for Monitoring, we can monitor the data of our sensor using its compatible app called ThingView (for IOS and Android phones download).
The internet communication protocol used is subject to the internet broker. For ThingSpeak, we can update values or send command via REST API or MQTT API. Both will have their own advantages over another you can check it out here. We will be using REST API protocol in this project. Today, let us begin with this project which uses smart phone app ThingView as a display to monitor real time data of an ambient temperature probe that I would like to monitor using ThingSpeak as broker. We will be using NodeMCU microprocessor board to execute all the programming.
NodeMCU micro-controller
The NodeMCU micro-controller is similar to Arduino micro-controller board. It is compatible with Arduino IDE software and is built-in with the ESP8266 module which can be connected to the internet. You can grab some at our affiliate link here !!!
Wireless Smart Energy Monitor for Saving
This is an energy meter for the whole household monitoring and the measurement unit is fit inside electrical panel while there is a portable monitor display which is constantly communicate and acquire data via wireless with the measurement transmitter unit. You can get it at our affiliate link here !!!
Wi-Fi Smart Energy Monitoring Control Meter
This is a smart household energy meter that could monitor and controlled by phone anywhere around the world at anytime. This is for single phase household and installed in electric panel via din rail. Get it at our affiliate link here !!!
ThingSpeak – Web Server
ThingSpeak is an open data platform and API for the Internet of Things (IOT) that enables you to collect, store, analyze, visualize, and act on data from sensors. Upon signup and register, it provides a dashboard or space for you to create unique channel IDs (which represents each project). For free version plan, you can create up to 4 channel IDs with each channel ID can store or monitor up to 8 fields (which represents each sensor or device readings). To sign up, you can go to https://thingspeak.com/. Once you have signup, you will go into the dashboard as below.
You firstly need to create a channel as demo for this project purpose. You may delete or modify later on for other purposes. There are few important things you need to note down: the channel ID, field (sensor) name, channel access: Public/Private, Write API Key and Read API Key.
The channel ID and Field ID are required to input in the arduino code so that the data can be sent to specific channel for display. Above is my Channel ID kindly use your Channel ID, not mine. Write API key are required for authentication from ThingSpeak when the Arduino tries to write the data to your channel while Read API key is required for ThingSpeak authentication when arduino tries to read data from your channel (to somewhere else) so that it could not be accessible by other person. Below is my sample of API Key and I have regenerated my key, kindly use your own API Keys.
In this project, let’s create and add the field 1 as ambient temperature sensor named “Temperature“.
Dupont line Wires
In order to connect wiring between Arduino board and module fast and easy, you need the dupont line cables. These cables are suitable during testing and troubleshoot where a lot of wire modifications. You can get it at our affiliate link here !!!
Ferrule Lugs and Crimper Set
Ferrule lugs are used at terminals for tight and secure connection. Besides, it also prevent stranded cables accidentally touching adjacent power line that may cause short circuit. Grab 1 at our affiliate link here !!!
ThingView Smartphone App
In order to monitor the Real Time Values and all historical values of your project from your smart phone, you can download and install its compatible app called ThingView. For IOS click here and for Android click here. Once you have created your ThingSpeak account, you need to link the ThingView App to your channel and you need the Channel ID, and Read API Key (if your channel is set private). Once you have keyed in the channel ID, all the fields value (measurement sensors) will automatically appeared. For any settings, you have to go to ThingSpeak and not much setting to be done in this app as it is just as display function.
UNI-T Multimeter is a good quality with decent price. We have been using UNI-T multimeter for years has not been any issue. We were using UT33C model. You can get yours at our affiliate link here !!!
You need the connector that can secure cables and isolate from accidental touch. Get the fast connector at our affiliate link here !!!
Hardware Wiring
As for this project example, I will be recording the ambient temperature of my room. I am using the NTC (Negative Temperature Coefficient) Temperature Probe. You may also need a 10k resistor in order for the micro controller board to measure the temperature value in analog voltage value. I will be using NodeMCU v3 micro controller board. There is a dupont cable connecting D0 and RST. It is for auto reset function if somehow the wifi could not be connected. Be sure to unplugged the cable during uploading code or else the code will fail to upload to the board.
Be sure that no conductive materials are placed around the workplace and use non-metallic enclosure or project box. Make sure wiring for all dupont cables are tight (for signal cable). Below is the overall wiring. It is quite simple and straight forward.
NTC Temperature Sensor
NTC Thermistor is a temperature sensor based on resistance value changes. The higher the temperature, the lower its resistance. You can grab it at our affiliate link here !!!
2 Relay Module
This is a relay module that can support 2 isolated wiring load control with each switching comes with NC and NO. Can support up to 10A for AC /DC loads. Controlable by NodeMCU or Arduino Board. Kindly get it at our affiliate link here !!!
Software Code
The final step would be adding source code onto NodeMCU board. I assume you have installed the Arduino IDE Software. If you have not installed the software, the link here can bring you to the official download site. Once you have downloaded the software, you may download the code file (.ino) for this application below (right click save link).
Before uploading the code to NodeMCU, if this is the first time you are dealing with NodeMCU, you need to add ESP8266 Support file and add NodeMCU board library into IDE software. To add ESP8266 Support file, open Arduino IDE Software, go to files>preferences>Additional Boards Manager URLs, paste this link http://arduino.esp8266.com/stable/package_esp8266com_index.json to the black space and click OK.
After finished this step, go to Tools > Boards > Boards Manager and download the ESP8266 library by ESP8266 Community
Once done, you can start download the .ino code file below and upload to the NodeMCU board. Before Upload the file, kindly unplug the dupont wire from pin D0 to RST pin or else code cannot be uploaded.
I will not display the code here because it is long. You can download the .ino file to see for your own. Almost all code lines are with explanation.
Download Data as excel file
In order to download the data, there are 3 locations can be download the recorded data. If you want to export recent data, you can click the button “Export Recent Data” on the channel page. It should able to provide data back calculated from now up to maximum 8000 data points per field with each data taken every 15 seconds.
If you want to collect all the data since the beginning of measurement, you can go to download button under “Data Import / Export” tab.
Both data download methods do not have much flexibility as both are fixed to time lapse of 15 seconds per data point. The 15 seconds per data point is too detailed if you are looking for a long term monitoring such as up to few years of measurement. Furthermore, ThingSpeak limits each download can collect up to 8000 data points which only could provide about 1.5 days of data and is too detailed to be analysed. If you think the same as I do, you may consider the 3rd method of download.
The final method is my favorite download method which using REST API command to your HTTP website to API web server of ThingSpeak so that they could send you the file for download. By this way you can fully customize which period you want to download with what time lapse between each data point you want. Of cause, the 8000 data points limitation is still there. But you can allocate data point margin to lengthen the measurement days. Below is the combination code for REST API command:
Code Structure :
http://api.thingspeak.com/channels/your channel ID/feeds.csv? condition1 & condition2 & condition3
“Your channel ID” is replaced by your Channel ID while condition 1, 2, 3 …are replaced by condition codes below.
Condition Codes:
For Example, my Channel ID is 992067 (public) and I want to collect data from date 2020-02-17 time 00:00:00 to 2020-02-18 time 00:00:00 (total 1 day) with time lapse (time scale :10 minutes per data point), my location is UTC +8, so the REST API code is :
http://api.thingspeak.com/channels/992067/feeds.csv?start=2020-02-17%2000:00:00&end=2020-02-18%2000:00:00×cale=10&offset=8
If my channel has changed to private access, I need to add the Read API Key. So the REST API code will become:
http://api.thingspeak.com/channels/992067/feeds.csv?start=2020-02-17%2000:00:00&end=2020-02-18%2000:00:00×cale=10&offset=8&api_keyXXXXXXXXXXXX
Just copy and paste the code to HTTP web browser and press enter. An excel file will automatically downloaded as below. Once you opened the file, all the data can then be analysed by convert it to graph or chart of your choice, it is then based on your creativity.
Before we end, we would like to give gratitude to you for taking the time to read the post. We would need readers like you to support us in order to keep growing. You can support us in the following ways :
Donate & Fund Raising
If you like my work, please send me a donation to encourage me to do more. Thanks
Aliexpress Affiliate
We are the member of Aliexpress affiliate marketing. Do support us by clicking the affiliate product links if you do wish to purchase them.
Like and Share
If you like our post, we need your support to like and share our posts or videos so that it can reach more and more people like you !!
Result – In ThingSpeak
Result – In ThingView
For Arduino Code Files, Remember to Right Click > Save Link As … You may alter the internal code as you wish. Happy coding !!
