Simple TCP readout and interface using pymodbus, sockets and flask
Quite often I find the need to have a simple display for showing the data coming from a modbus TCP interface. You can use a variety of software to present this information, however most of the time you are limited to just reading the registers with no additional transformations applied to the information.
There are several libraries and software capable of pulling all the data, ranging from a celery systems to a kafka implementation. But sometimes I just want something quick to figure out if the sensor is putting formation out that I can check form a website.
To do so I turn to three main libraries: pymodbusm threading and flask (and flask-socketio).
Setting up our test devices
In order to test this out I'll just create a simple TCP server with pymodbus and docker. So I can test it out, this is a modification of one of the examples provided in the pymodbus example website, which is an excellent resource. Pymodbus Example.
# import the modbus libraries we need
from pymodbus.server.asynchronous import StartTcpServer
from pymodbus.datastore import ModbusSequentialDataBlock
from pymodbus.datastore import ModbusSlaveContext, ModbusServerContext
#We use twisted for the the continues call, could also use a thread for updating
#the values
from twisted.internet.task import LoopingCall
#This is the function that will be called periodically
# we define which registers and to read and modify
def updating_writer(a):
context = a[0]
register = 3
slave_id = 0x00
address = 0x00
#Read the current values
values = context[slave_id].getValues(register, address, count=5)
values = [v + 1 for v in values]
#Increase the current values
context[slave_id].setValues(register, address, values)
#this actually setup our server and the content of the registers across
# we set the context to single as we only have the one slave context
def run_updating_server():
store = ModbusSlaveContext(
di=ModbusSequentialDataBlock(0, [17]*100),
co=ModbusSequentialDataBlock(0, [17]*100),
hr=ModbusSequentialDataBlock(0, [17]*100),
ir=ModbusSequentialDataBlock(0, [17]*100))
context = ModbusServerContext(slaves=store, single=True)
#Define how often the update the values and start the server
time = 5 # 5 seconds delay
loop = LoopingCall(f=updating_writer, a=(context,))
loop.start(time, now=False) # initially delay by time
StartTcpServer(context, address=("0.0.0.0", 5020))
if __name__ == "__main__":
run_updating_server()
This will increase the value in the first 5 holding registers by one every 5 seconds. The setup of server just sets up all the discrete inputs, coilds, holding registers and input registers to hold.
In order for this to work you have to set the address to be "0.0.0.0" not localhost and the port to be 5020.
Our dockerfile looks like this:
FROM python:3.7-slim-buster
RUN mkdir /app
WORKDIR /app
COPY . .
RUN pip install pymodbus
RUN pip install twisted
CMD ["python","updating_server.py"]
EXPOSE 5020Then we will build it with docker build -t tcpTestServer .
Now that the image is build we can just run the following TCP server to spin up two test TCP servers on different ports with test data.
version: '3'
services:
machine1:
image: tcpTestServer:latest
ports:
- 5021:5020
machine2:
image: tcpTestServer:latest
ports:
- 5022:5020
Now we can run our containers using docker-compose up this will spin up the two containers and they will pass information on port 5021 and 5022.
Getting values from a TCP server with pymodbus
Pymodbus makes it pretty easy to read values from a tcp server. In order to do so all we need to do is define a ModbusTcpClient with the IP address and port of our tcp server.
from pymodbus.client.sync import ModbusTcpClient
client = ModbusTcpClient(ip_address, port)
now if we we want to read a coil we can just read it like so
client.read_coils(start, end, unit=unit_num)
to read a holding register we would use
client.read_holding_registers(address, count)
More likely than not we will have do some decoding depending on how our data is being transmitted, depending on doubles and similar. Generally I end up using the BinaryPayloadDecoder included in pymodbus. To use it we will need to declare the byteorder and the wordorder.
from pymodbus.constants import Endian
from pymodbus.payload import BinaryPayloadDecoder
read=client.read_holding_registers(address,count)
#Decoding the first 2 registers
data_encoded=BinaryPayloadDecoder.fromRegisters(read.registers[0:2],
byteorder=Endian.Big,wordorder=Endian.Little)
data_decoded=data_encoded.decode_32bit_uint()
For this project we wont really have to decode the values as it fits in a single register.
Now that we have a way to read our servers we can move it to the background using threads, which we can use to read multiple servers, without really increasing the complexity of the code too much (but you probably want to set up workers and such instead).
Creating the Flask Display
Flask is awesome. There are a lot of people that are significantly better than me at it. However I makes it pretty easy to write microservices, small websites and api's using a variety of plugins. Plus it takes jinja templates to do quite a bit of magic.
from flask import Flask, render_template, url_for, copy_current_request_context
# Initialize the app
app=Flask(__name__)
app.config['SECRET_KEY'] = 'secret!'
app.config['DEBUG'] = True
# Define a simple route to our index, and use our index html
@app.route('/')
def index():
return render_template('index.html')
#Set the ports and ip on where to run
if __name__ =='__main__':
app.run(host='0.0.0.0',port=5000,debug=True)
Now we need to define a template for this:
</head>
<body>
<div class="container">
<div class="jumbotron">
<h1>Device 1</h1>
<p>Device Live Feed</p>
</div>
</div>
</div>
<div class="container" id="content">
<div class="row">
<h3>Latest Readings:</h3>
<div id="log">
</div> <!-- /#log -->
</div>
</div>
</body>
</html>
Moving ModBusTcp Reads into a thread
We really don't want to hold up Flask or the program with the reads to the Tcp server. So we will move the read out to a thread running in the background.
from flask import Flask, render_template, url_for, copy_current_request_context
from threading import Thread, Event
from pymodbus.client.sync import ModbusTcpClient
from pymodbus.constants import Endian
from pymodbus.payload import BinaryPayloadDecoder
import queue
import time
app = Flask(__name__)
app.config['SECRET_KEY'] = 'secret!'
app.config['DEBUG'] = True
We will need to define a class inheriting from Thread, and initialize our modbus client. We also declare an event that we will clear when we want to stop the thread by overriding the join Thread method later on.
class ReaderThread(Thread):
def __init__(self, ip_address='0.0.0.0',port):
Thread.__init__(self)
self.alive=Event()
self.alive.set()
#define and initialize our client
self.client = ModbusTcpClient(ip_address,port=port)
Within the class we want to declare a method to get the data and to run the thread.
def read_device(self):
client=self.client
read=False
data=0
data_read=client.read_holding_registers(0x00,1)
data=data_read.registers[0]
read=True
return (read,data)
def run(self):
try:
self.client.connect()
except:
print('not connected')
while self.alive.isSet():
time.sleep(1)
read=False
try:
read, data = self.read_device()
except:
print('lost connection')
if read:
print(data)
if self.client:
self.client.close()
def join(self):
self.alive.clear()
Thread.join(self)We also need to start the initialize a ReaderThread in the code so we put that before defining our routes and set it to run as daemon.
uThread=ReaderThread()
uThread.daemon = True
Socket IO Implementation
Now that we have this setup we in theory could create an API to call from JS from the website. However since I'm lazy and just want to keep this simple. We can just use a socket. Flask has an addon for this flask_socketio. To get started we change our code to the following
from flask_socketio import SocketIO, emit
from flask import Flask, render_template, url_for, copy_current_request_context
from threading import Thread, Event
from pymodbus.client.sync import ModbusTcpClient
from pymodbus.constants import Endian
from pymodbus.payload import BinaryPayloadDecoder
import queue
app = Flask(__name__)
app.config['SECRET_KEY'] = 'secret!'
app.config['DEBUG'] = True
#turn the flask app into a socketio app
socketio = SocketIO(app)We change our class run declaration to the following, we could add a namespace to emit to different namespaces depending on device if running more than one :
def run(self):
try:
self.client.connect()
except:
print('not connected')
while self.alive.isSet():
time.sleep(1)
read=False
try:
read, data = self.read_device()
except:
print('lost connection')
if read:
socketio.emit('newvalue',{'number':data})
if self.client:
self.client.close()
Now we have to initialize flask socket io. Set the behavior when we connect (if the thread is not alive start it).
app = Flask(__name__)
app.config['SECRET_KEY'] = 'secret!'
app.config['DEBUG'] = True
#turn the flask app into a socketio app
socketio = SocketIO(app)
.
.
.
.
uThread=ReaderThread()
uThread.daemon = True
@app.route('/')
def index():
return render_template('index.html')
@socketio.on('connect')
def test_connect():
global uThread
# access the global thread. If the thread isn't already up
# create a Thread object and start it (again running as daemon)
if not uThread.isAlive():
print("Starting Thread")
uThread=ReaderThread()
uThread.daemon = True
uThread.start()
print('Client connected')
print("Starting Thread")
@socketio.on('disconnect')
def test_disconnect():
print('Client disconnected')
if __name__ == '__main__':
socketio.run(app, host='0.0.0.0', port=5000, debug=False)
Last piece of the puzzle is to modfiy our index.html template to actually call and write our recived values from the socket. So we modify it to append the new value to our array when the 'newvalue' that we emit from to the socket is recivied.
<!DOCTYPE html>
<html>
<head>
<script src="//code.jquery.com/jquery-3.3.1.min.js"></script>
<script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/socket.io/1.3.6/socket.io.min.js"></script>
<script type="text/javascript">
$(document).ready(function(){
//connect to the socket server.
var socket = io.connect('http://' + document.domain + ':' + location.port);
var values_received = [];
//receive details from server
socket.on('newvalue', function(msg) {
console.log("Received Value" + msg.number);
//maintain a list of ten numbers
if (values_received.length >= 15){
values_received.shift()
}
values_received.push(msg.number);
value_string = '';
for (var i = 0; i < values_received.length; i++){
value_string = value_string + '<p>' + values_received[i].toString() + '</p>';
}
$('#log').html(value_string);
});
});
</script>
<link rel="stylesheet" href="//maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css">
</head>
<body>
<div class="container">
<div class="jumbotron">
<h1>Device 1</h1>
<p>Device Live Feed</p>
</div>
</div>
</div>
<div class="container" id="content">
<div class="row">
<h3>Latest Readings:</h3>
<div id="log">
</div>
</div>
</div>
</body>
</html>
So now we should have a nice website to look at our readings that I should be able to access form my web browser, I normally just keep it that way since I access from within a VPN. But you can move it over to gevent pretty easy.
from tcp_reader import app
from gevent.pywsgi import WSGIServer
server=WSGIServer(('0.0.0.0', 5000), app)
server.serve_forever()
Full code over here.