IoT - Developing a Java library to switch on/off TP-Link Tapo P100 Wifi wall plugs/lights for my Domotica apps

Author: Jan Willem Teunisse, 28 November 2021 (edited 28-12-2021)

1. Introduction

This article describes the development of a Java JAR library in order to switch on/off my WiFi TP-Link Tapo P100 wall plugs. I recently bought a couple of these P100 wall plugs and a L510E light bulb to be used with my domotica system.

Normally these Tapo plugs or light bulbs are controlled with an Android app on my smartphone. But I like to control the devices also by using my self developed Domotica system and another Android app. This app I have developed by using a cross RAD development platform called B4X(B4J, B4A, B4I and B4R) from Anywhere Software.

After some search efforts on the internet I found some github implementations of software how to get things done. These implementations were written in Python, Java, Golang and based on reverse engineering. As my own Android app is Java based, I started with a Java version.

Reference material for further reading:

  1. An article of a reverse engineering effort is described in this link tp-link-tapo;
  2. His Proof of Concept you can find in this link.There you can also find an Python version ;
  3. A Python implementation is made by fishbigger, see his Github link Tapo P100 Python library.
  4. The Java one I adopted is from com.adeptues.p100, see the link adeptues.p100.
  5. The Golang version you can find here.
After finishing the Java JAR library, I continued by adopting the Golang version (ref. 5) in order to use it for a module written in Go.

2. Structure of connecting to the Tapo devices

2.1 Initializing a Tapo device

The Tapo app uses two sorts of communication:

  1. First to install and to configure the new unpaired Tapo devices with a Bluetooth connection, using variables as IP-address, Wifi SSID, the device nickname, etc. Also an emailaddress and password is required;
  2. After the initial pairing step, the app uses HTTP requests to switch the Tapo device, get the device information, etc.
After the first initial step I gave each Tapo device a static IP-address by setting this in my Wifi router.

2.2 The Development part

Using the Windows version of the B4J IDE I started with a port of fishbiggers Python version by using the Basic language, which the B4J or B4A IDE's uses.
First I made a key-value list or map of the three devices or nodes, as I called them in relation with my Zwave nodes. The key-value map looks a bit like an INI file.


[nodes]
tapo_comment=tapo wifi plugs configuration file
tapo_user=tapo@email.com
tapo_pw=my_tapo_password
tapo_count=3
tapo0=name;ip_adress;    mac_adres;        type;model;light%;zone;status <-- this is a comment line
tapo1=test1;192.168.x.n1;E8-48-XX-XX-XX-XX;plug;P100;#;study;#
tapo2=test2;192.168.x.n2;E8-48-YY-YY-YY-YY;plug;P100;#;study;#
tapo3=test3;192.168.x.n3;E8-B6-ZZ-ZZ-ZZ-ZZ;light;L510E;50;outside;#

The emailaddress and password are the same as used in your smartphone Tapo app.

The Tapo HTTP request's uses the information stored in this map like the email-username/password, the IP- and MAC-Address. For testing purposes I use besides a B4J test application also a Groovy script tapoaction, which can later be used as a command line program, to be called from my Windows domotica server.
The command line looks like:
prompt> tapoaction device_name action brightness
An example:
prompt> tapoaction test3 on 50

Instead of the word on also true can be used. The word off or any other word can be used to turn the device off.

The HTTP requests use a lot of base64 encoding and encryption. This turned out to be rather difficult for me to translate all these Python code in something that could be used in B4J/B4A code module. These IDE's supports also Java JAR libraries. So I turned to the Java implementation(s) of fishbiggers Phyton code in order to develop my own version of a Java JAR file.

In B4J/B4A a library has first to be initialized. In this first method the following class variables are set like encodedEmail, encodedPassword, a private/public keypair, the IP-Address, the TerminalUUID (MAC-address), the later received token and cookie. Also other variables are defined like errorCode and the last errorMessageText.

The used (private) HTTP request functions are:
- handshake()
- login()
- switchOnOff()
- setBrightness()
- getDeviceData

The public methods to communicate with the device and my Android app or Groovy script are:
- PlugSwitchOnOff()
- SetLightBrightness()
- GetDeviceInfo()

The Java code was developed using Notepad+ editor on my Windows PC.
The library package is called nl.jwtipa.tapop100 and contains two classes:
- public TapoP100
- public TPLinkCipher

Screen B4J Simple Library Compiler In order to compile the Java Library codes Anywhere Software supports a tool, called LibraryCompiler.exe. After a succesfull compilation the Java JAR file is build and placed in a folder ...\dev\B4X\AdditionalLibraries with an Javadoc XML file containing the Help information how to use the various methods.

The development process was done step by step, mostly because my old school Java coding knowledge was very rusty and also a lot off the used imported classes gave me a lot of compile errors. And I had to program some workarounds or simplifly the used code in an other way. But in the end I succeeded in getting the Tapo plugs and light bulb to switch on/off and setting the brightness of the LED light.

The first step was to develop the initialisation code like the encodig/encrypting of the variables emailUser and emailPassword, followed by the private/public key pairing. After that the second step was developing and testing the handshake method. In this step I simplyfied the code for secure payload due to errors in not finding the correct JAR file on the internet. After testing the handshake() method with the life P100 wall plug and getting back a JSON string with no errrors, it was time to develop the 3rd step the login() method. And soforth the methods switchOnoff, setBrightness and in the end getDeviceData.

2.3 Testing step by step using B4J app

With step by step development I succeeded in a version that works in the sense that plugs are reacting with on/off click sounds of the relays and the light bulb reacted on the various brightness settings. Although I still had many warnings (about 52) during compilation and strange errorCodes in the testing phase.

2.3.1 Testing the (prototype) code

The following code block shows the part of the B4J basic code to test the different Java methods. tekst


....
	Dim m As mdf     ' mdf = micas defined functions library
	Dim u As udf     ' udf = user defined functions
	m.Initialize
	u.Initialize
	' retrieving Tapo nodes configuration settings from file taponodes.map
	Dim TapoNodes As Map
	If (File.Exists(File.DirApp, "taponodes.map")) Then
		TapoNodes = File.ReadMap(File.DirApp, "taponodes.map")
	Else ' something is wrong, stop the app
		Log("Can not read the file 'taponodes.map'")
		' stop app
		Log("FirstProgram Exit")
		MainForm.Close
		ExitApplication
	End If
	TapoUser = TapoNodes.Get("tapo_user")
	TapoPassWrd = TapoNodes.Get("tapo_pw")
	NrofTapoNodes = TapoNodes.Get("tapo_count")
	TapoNodeList = m.GetTapoNodeListByName("test2", TapoNodes)
	Log("tapo List = " & TapoNodeList) 
	TapoIP = m.GetTapoIPByName("Zijdeur", TapoNodes)
	TapoMac = m.GetTapoMACByName("Zijdeur", TapoNodes)
	' testing TapoP100 methods like PlugSwitchOnOff
	Dim tp As TapoP100  ' use the java library
	tp.Initialize(TapoIP, TapoUser, TapoPassWrd, TapoMac)
	Dim tekst As String = tp.MsgText
	Log("msgtext: " & tekst)
	Dim DeviceInfo As String = tp.GetDeviceInfo(TapoIP, "Zijdeur")
	Log("Device Info: " & DeviceInfo)
	Dim intStatus As Int = 0 
	intStatus = tp.SetLightBrightness(TapoIP, "Zijdeur", "25", True)
	If intStatus < 0 Then
		Log("Brightness Status < 0: " & tp.MsgText)
	Else
		Log("Brightness Status: " & tp.MsgText)
		Log("Cookie: " & tp.GetCookie)
	End If
	intStatus = tp.SetLightBrightness("192.168.x.nn", "Test3", "0", False)
	If intStatus < 0 Then
		Log("Brightness Status < 0: " & tp.MsgText)
	Else
		Log("Brightness Status: " & tp.MsgText)
		Log("Cookie: " & tp.GetCookie)
	End If
	intStatus = tp.PlugSwitchOnOff("192.168.x.mm", "Test2", True)
	If intStatus < 0 Then
	  Log("SwitchStatus < 0: " & tp.MsgText) 
	Else 
	  Log("SwitchStatus: " & tp.MsgText)
	  Log("Cookie: " & tp.GetCookie)
	End If
	DeviceInfo = tp.GetDeviceInfo("192.168.x.mm", "Test2")
	Log("Device Info: " & DeviceInfo)
	intStatus = tp.PlugSwitchOnOff("192.168.x.mm", "Test2", False)
	If intStatus < 0 Then
		Log("SwitchStatus < 0: " & tp.MsgText)
	Else
		Log("SwitchStatus: " & tp.MsgText)
		Log("Cookie: " & tp.GetCookie)
	End If
....

Although switching the plugs or light bulbs functions correctly, there are still warnings and some errors in the Java library code. As you can see in the following console print-out of the B4J IDE. And that's probably due to the many compile warnings I still get.


Waiting for debugger to connect...
Program started.
FirstProgram is started
tapo List = Zijdeur;192.168.1.x;E8-48-YY-YY-YY-YY;plug;P100;#;outside;#
msgtext: initialized
Device Info: model=L510 Series;device_on=false;on_time=0;nickname=Zijdeur;
Brightness Status < 0: JSONException, etc. setBrightness encrPayload
Brightness Status < 0: extractToken errorCode -12, Unknown error -12
SwitchStatus < 0: extractToken errorCode -12, Unknown error -12
Device Info: model=P100;device_on=true;on_time=12;nickname=P100-2;
SwitchStatus < 0: extractToken errorCode -12, Unknown error -12
.....

So this is still work in progress. I will update this article along the way....

2.4 Testing using a Groovy script TapoAction

To see if the Java JAR can also be used in some of my Groovy domotica scripts, I developed a Groovy script called tapoaction.groovy

Therefore I copied the JAR file in my %GROOVY_HOME%\lib folder, and started to write the following script.


package nl.jwtipa.tapoaction;
/*
 *  comments 
*/
import groovy.transform.Field;
import nl.jwtipa.tapop100.*;
public static void main(String[] args) {
  ...
  // code to read the tapoNode.map settings
  ...
  TapoP100 tp = new TapoP100()
  tp.Initialize(IPAddress, emailUser, emailPW, MACAddress) ;
  if (ModelType == "P") {
    errorCode = tp.PlugSwitchOnOff(IPAddress, NodeName, bOnOff) ;
  } else { // light bulb
    errorCode = tp.SetLightBrightness(IPAddress, NodeName, Brightness, bOnOff) ;
  }  
  if (errorCode < 0) {
    LogToFile(LOGFILENAME, "Error MsgText : " + tp.MsgText) ;  
  }
} // end of main
} // end of class TapoAction

After testing this script, I compiled the groovy source code to a JAR file in order to deply it on my Windows domotica server. The groovy script works om my development PC, but on my target domotica Windows server it won't not start yet. Again work to be done.....

2.5 Fine tuning the Java code

In order to get rid off the many compile warnings, I have to find a way to deal with this. First I have to find a method of compiling the Java library using the Java compiler instead of the Simple Library compiler of B4J.

2.6 Conclusion and download the Java source codes

It was fun to develop my own version of the Java TP100 library, although it needed a lot trial-and-error hours and researching the internet in how other developers handled several (error) situations.

If you do like to develop your own Java library, you can download a ZIP file containing the Java source code files: TapoP100.java and TPLinkCipher.java. For me it is not ready yet to put it on my Github account.

In the following chapter you can read my efforts in developing a Go(lang) version of the TP100 library.

Back to top

3. The Golang TapoP100 module tpdf

3.1 Introduction

After developing a workable Java version, I focussed on developing a Go(lang) package, called tpdf (short for Tapo Defined Functions). This Go version is based on the Github source code mentioned in Ref. [5].
The original package is extented with some extra error checking, f.i. if you use een light brightnes setting of zero (0) value, we get an error -1008 back. The same happens with a value greater then 100. A value greater than 100 is reset to a value of 100 in the SetLightBrightness() function.

Also after you have performed the login sequence you have a time frame of 24 hours to switch on or off the plugs or the lights. When you try a new handshake or a login in this time frame, you get an error 9999. During testing I also noticed that the light was not always responding, so I included in my commandline tool tapoaction a pinging function to detect if the plug or light was responding. Apparently you have sometimes to wake-up the light bulb.

Back to top

Testing using a Go TapoAction program

To see if the Go TP100 package can also be used in some of my Go domotica executables, I developed a Go command line program, also called tapoaction

The program extracts the commandline arguments like ..
prompt> tapoaction test3 on 50

Then the tapnodes.map file is read and from the plug (or light bulb) name the IP-address and MAC-address are extracted from taponodes map. With these data the tapo "device configuration" is initialized and the plug or light bulb can be switched on (or off).


package main
//  program: tapoaction - executing autodomus action(s) for TP-Link Tapo P100/L105 devices
//  Author:  J.W. Teunisse
//  Version: 0.0.5  edited 28-12-2021
//  testing Go
//  prompt:> tapoaction.exe nodename on_off [brightness]
// 
//  nodename	name of the Tapo TP-Link device (nickname)
//  on_off	   value, f.i. on|off, true|false, .
//  brightness value of brightness	0..100 % 
//  
//  Example: tapoaction.exe test3 on 15 
//
//  Exit codes:
//  0  = succesfull, no errors
//  1  = no commandline arguments
//  2  = no device nodeName found in tapoNodesMap
//  3  = no taponode.map file found in the \exes directory
//  4  = errors found during execution of the HTTP requests
//  5  = error during ping trial
//
// Golang code on the basis of https://github.com/artemvang/p100-go
//
//  to do:
//  - cleanup println test loglines
//  - check ping in a loop van 3 times with 1 seconde sleep

import (
//    b64 "encoding/base64"
   "bufio"
//	"bytes"
    "fmt"
//	"io/ioutil"
    "os"
	"net"  //http"
	m "mdf"
   "tpdf"
	s "strings"
	"strconv"
	"time"
//	"log"
)

const VERSION = "V0.0.5"
const LOGFILE= "tapoaction.log"
const LOGDIR = "D:\\AutoDomus\\logs\\" ;
const TAPOMAP_FILENAME = "taponodes.map"

func main() {
  var emailUser, emailPW, IPAddress, modelType, tapoMAC string
// var subsys string
 var nodeName string
 var sOnOff string
// var vandaag string
 var Brightness, defaultBrightness string
 var key, model, deviceInfo, deviceName, deviceType, waarde string
 var adDir string
 var exeDir string
 var logDir, logLine string
 var tapoMapFilename string
// var ZWaveUrl string
// var delay int
 var i, xL int  //, l, p int
// var NrOfPlugs int = 0
 var ok int
 var bOnOff bool
// nodeid = "2"
 sOnOff = "on"
// bOnOff = true

// os.Args provides access to raw command-line arguments. 
// Note that the first value in this slice is the path to the program, and os.Args[1:] holds the arguments to the program.
//  argsWithProg := os.Args ;   argsWithoutProg := os.Args[1:]
// You can get individual args with normal indexing.
  appName := os.Args[0]
  appName = s.TrimSpace(appName)
  fmt.Println("Starting Autodomus Action (",appName,") ",VERSION," ....")
 
  // waarde = os.Args[5]
  n := len(os.Args)  // check number of arguments
  if (n >= 3) {
    nodeName = os.Args[1]
    sOnOff = s.ToLower(os.Args[2])
    if (n > 3) {
      Brightness = os.Args[3] ;
    } else {
      Brightness = "50"   // set to default 50%    
    }  
  } else if (n == 1) {
    fmt.Println( " No arguments or commandline parameters are given:\r\n command prompt: tapoaction.exe nodename action brightness\r\n\r\n Program stops here.\r\n\r\n tapaction version ")
    nodeName = "Zijdeur";  // plug
    modelType = "L"
    sOnOff = "true" 
    Brightness = "25" ;
    fmt.Println("  Node: ", nodeName, sOnOff, Brightness)
    os.Exit(1) 
  } // end of test args
  
  if (sOnOff == "true") {
    bOnOff = true 
  } else if (sOnOff == "on") {
    bOnOff = true ;
  } else {
    bOnOff = false ;
    sOnOff = "off" ;
  }  
  
  adDir = os.Getenv("AUTODOMUS")
  exeDir = adDir+"\\exes"
  logDir = adDir+"\\logs"
  // fmt.Println("  env Autodomus: ", adDir, ", exe_dir: ", exeDir, ", log_dir: ", logDir) // printout for tests
  ok = m.LogToFile(logDir, LOGFILE, "Starting Autodomus Action (" + appName + ") "+VERSION+" ....")
  if ok != 1 { fmt.Println("  No logline appended") }
  logLine = "  execute command for "+nodeName+": "+sOnOff  // +" "+nodeid+" "+action  
  ok = m.LogToFile(logDir, LOGFILE, logLine)
 
  // read in taponodes.map config file
  tapoMapFilename = exeDir + "\\" + TAPOMAP_FILENAME  
  //  fmt.Println("TapoMap file: ", tapoMapFilename)
  tapoNodesMap := make(map[string] string)
  //  tapoNodesMap["tapo_user"] = "mail@domein.com"
  //  tapoNodesMap["tapo_pw"] = "password" 
  // fmt.Println("TapoMap: ", tapoNodesMap)
  x := "" // file map regel
  var mapLine []string
  fhMap, err := os.Open(tapoMapFilename)
  if err != nil {
      errLine := "  Failed opening input file: "+tapoMapFilename
		fmt.Println("  "+errLine, err)
      ok = m.LogToFile(logDir, LOGFILE, errLine)
      os.Exit(3)  // 3 = no TapoMapFile found
  }
	scanner := bufio.NewScanner(fhMap)
	scanner.Split(bufio.ScanLines)
	for scanner.Scan() {  // verwerk de file regels
      i = i + 1 
		x = scanner.Text()
      // x = u.StrTrimTrailing(x, ",")
      // x = u.StrTrimTrailing(x, ";")
      // x = u.StrTrimTrailing(x, "\t")      
      xL = len(x)
      if (xL > 0) {
        if s.Index(s.ToUpper(x), "[NODES]") == 0 {  // check on [Nodes] part
          continue
        }
        mapLine = s.Split(x, "=")        // process map line
        key = mapLine[0]
        waarde = mapLine[1]
        tapoNodesMap[key] = waarde
      } // end if xL > 0  
	} // end scanner
	fhMap.Close()   
   emailUser = tapoNodesMap["tapo_user"]
   emailPW = tapoNodesMap["tapo_pw"]
   waarde = tapoNodesMap["tapo_count"]
   n, _ = strconv.Atoi(waarde)
   // fmt.Println("  Tapo Values: ", emailUser, emailPW, waarde)
   // get nodename info like IPAddress, etc
   for i := 0; i <= n ; i++ {
     key = "tapo" + strconv.Itoa(i)
     x = tapoNodesMap[key]
     mapLine = s.Split(x, ";")     
     deviceName = mapLine[0]
     if (s.ToUpper(nodeName) == s.ToUpper(deviceName)) { // bingo
       IPAddress = mapLine[1]
       tapoMAC = mapLine[2]
       deviceType = mapLine[3]
       model = mapLine[4]
       modelType = model[0:1]
       defaultBrightness = mapLine[5]
       i = n+1 // stop loop
     } else {
       x = "? No device values found for Tapo device "+nodeName
       deviceName = "?NF"  // "? not found"       
     }
   } // end of get nodeName information   
    fmt.Println("  Tapo Device Info: ", key, deviceName, x, IPAddress, tapoMAC, deviceType, model, modelType, defaultBrightness)
   // check on nodeName found, if not then stop with an exit.
   if deviceName == "?NF" {
     logLine = "  " + x + "\r\n  execution stops.."
     fmt.Println(logLine)
     ok = m.LogToFile(logDir, LOGFILE, logLine)
     os.Exit(2)  // 2 == no nodeName found     
   }
   //check if with IPAddress of Tapo Device is alive, try max. 3 times
    checkHost := IPAddress + ":80"
    timeout := time.Duration(1 * time.Second)
    for n=1; n<4; n++ {
      _, err = net.DialTimeout("tcp", checkHost, timeout)
      if err != nil {
        if n >= 3 {
          logLine = fmt.Sprintf("%s %s %s\n", checkHost, "not responding", err.Error())
          fmt.Println("  ", logLine, "\n tapoaction stops ....")
          ok = m.LogToFile(logDir, LOGFILE, logLine)
          os.Exit(5) // no reaction from Tapo device
        } else {
          time.Sleep(1*time.Second)  // wait and try again
        }           
      } else {
        logLine = fmt.Sprintf("  %s %s\n", checkHost, "responding on port: 80")
        fmt.Println(logLine)
        n = 4 // stop this for loop        
      } 
   } // end for loop      
   
   deviceCfg := tpdf.Initialize(IPAddress, tapoMAC, emailUser, emailPW)   // set device configuration
   // fmt.Println(deviceCfg)
   if modelType == "P" {
     fmt.Println("  Start PlugSwitchOnOff")
     ok = deviceCfg.PlugSwitchOnOff(nodeName, bOnOff)
   } else {
     fmt.Println("  Start SetLightBrightness")
     ok = deviceCfg.SetLightBrightness(Brightness, bOnOff)
   }
   fmt.Println("  returnstatus: ", ok)
   fmt.Println("  switch status: ", deviceCfg.GetValue("status"))   // string method
   if ok < 0 {  // log eventual errors
     logLine = "Error: "+ tpdf.GetErrorMsg(ok)
     ok = m.LogToFile(logDir, LOGFILE, logLine)  
   }

   deviceInfo = deviceCfg.GetDeviceInfo()
   fmt.Println("  device info: ", deviceInfo)

  ok = m.LogToFile(logDir, LOGFILE, "Ending Autodomus TapoAction.")
} // end of main.

The output looks like this for my Zijdeur (side door) LED light with a 25% brightness...


D:\dev\Go\src>tapoaction Zijdeur on 25
Starting Autodomus Action ( tapoaction )  V0.0.5  ....
  Tapo Device Info:  tapo3 Zijdeur Zijdeur;192.168.nn.nn;D8-07-AA-BB-CC-DD;light;L510E;50;buiten;# 192.168.nn.nn D8-07-AA-BB-CC-DD light L510E L 50
  192.168.nn.nn:80 responding on port: 80

  Start SetLightBrightness
  returnstatus:  1
  switch status:  on
  Start getDeviceData
  device info:  model=L510 Series;device_on=true;on_time=0;nickname=Zijdeur;

3.2 Download the Go(lang) source codes

If you like to develop your own Go TP100 package, you can download a ZIP file containing the Go sources codes.

Back to top

4. Summary

The main goal in this article was to share my knowledge in porting and developing these two types of libraries.

You can find the source codes here in a ZIP file.
- Java version: source code TP100
- Go version: source code TP100

5. Licenses and copyright

Licenses of the used software components.


© Copyright 2021/2022 by J.W. Teunisse
This piece of software as presented in this article is provided 'as-is', without any express or implied warranty. In no event will the author be held liable for any damages arising from the use of this article or software (code).

Back to top

6. Comments or advice

Your comments or advice for improvement are most welcome, you can send them to the following email-address pr@jwteunisse.nl

Back to top