ESP32 HMI 4.3-inch Arduino Tutorial¶

Requirements for the relevant version of the routine
Board Version
esp32 by Espressif System 3.3.8
Lib Related Versions
Ivgl: 8.3.11
GFX Library for Arduino: 1.6.5

The libs are provided directly by our wiki, just use the libs provided by the wiki.

Overview¶

This example tutorial is a project for controlling the on and off of an LED. It demonstrates how to create a UI and how to control the on and off of an LED through buttons on the screen.

In this tutorial, we will show you how to design the UI with SquareLine Studio, and show you how to upload the code with Arduino IDE.

Hardware Preparation¶

CrowPanel ESP32 4.3'' HMI	Crowtail-DHT20	Crowtail-LED

Design UI file with SquareLine Studio¶

Get Started with SquareLine Studio¶

Please click the card below to learn how to download the SquareLine Studio, and how to export the demo UI file.

Design UI file with SquareLine Studio¶

Let's start learning how to create our own UI after getting an initial understanding of SquareLine Studio.

Open the SquareLine Studio and create a project.

A. In the latest version of SLS, select "Elecrow"->"DIS06043H...Arduino-IDE"

B. In the earlier version of SLS, select "Arduino"->"Arduino with TFT_eSPI".

Note：

When select the Arduino framwork, there's only one option "Arduino with TFT_eSPI". By choosing this, the squareline studio will generate a template code suitable for the TFT_eSPI library. However, squareline studio not only supports the TFT_eSPI library, it supports a variety of libraries to suit different hardware and application needs. For example, Adafruit_GFX library, LovyanGFX etc.

After using SLS to generate UI code, we then use different graphics libraries according to different hardware and modify the corresponding code to display the content you design.

Set the name of the project, set the screen resolution to 480*272, set the color depth to 16bit, and keep other default settings. After setting, click CREATE to create the project.
- 16-bit color depth: can represent 65,536 colors through RGB 5:6:5 sub-pixel representation, that is, each RGB channel occupies 5 bits and 1 bit (a total of 16 bits) to represent colors.
After creation, enter the following interface with a blank background.
In the "Assets" area, click "ADD FILE TO ASSETS" to add custom images or icons.

Please click to download the custom images used in this tutorial.

Note：

Images only support webp format. The pixels of the image need to be smaller than the pixel size of the screen used in your project. The size of each image should not exceed 100k, preferably within 30k, to provide a smooth display effect.
Add background.

Find "Inspector"->"STYLE SETTING", click to expand "STYLE(MAIN)", then click the 2^nd "Background". Check the "Bg Image" and select the background image.
Add Button widget to control the LED.

Click "Button" in the "Widgets" area, and "Button1" will be added to the current Screen.

The position and size of the label can be adjusted by dragging the mouse. You can also directly enter numbers in the Inspector→BUTTON→Transform to adjust.

Add an identification symbol to the button. The button in this tutorial controls the LED switch, so you only need to mark the button "on" and "off". You can add LABEL widgets or add a background images to the button. This tutorial will demonstrate how to add a background image to a button.

Click the Button1, then find Inspector->STYLE SETTINGS ->STYLE(MAIN) ->Background, and select the image.

In the same way, duplicate a Button widget. And drag it to the corresponding position to modify different background image.

Set the status of the button to identify different states.

In "Inspector"->"STYLE SETTINGS"->"STATE", set display white background color by DEFAULT and red when on the PRESSED state.

Make the same settings for the "OFF" button.
Add events to buttons.

Note: Because the button controls the on and off of the LED, we can add any event here to generate the code framework for the button event when exporting the UI file. We will modify the code of the button event to control the LED latter.

Select the button and click "ADD EVENT".

Select "released" as the trigger condition, select a trigger event in "Action". It will be modified in the generated program to achieve the LED control function.

Complete the event. Here I choose to change the screen, and the screen to be switched is Screen1.

Add event to Button2 (OFF) in the same way.
Export UI files.

Click "File" -> "Project Settings" and make settings for the exported file.

Set the export path of the file (set the path according to your own file).

Fill in lvgl.h in LVGL Include Path. Check "Flat export(exports all files to one folder )".

Then click "APPLY CHANGES".

Tips: After selecting the flat export, the output files will be in the same folder, so that the output code does not need to modify the path in the program. If not, the output files will be classified and placed in different folders. The compiler may not be able to recognize different paths, which will cause some trouble. In this case, the user needs to modify it manually, so it is recommended to select all files to be output to the same folder.

Export UI files. The exported files will be in the path we set earlier.

In order to be used with the main program, all UI files generated need to be placed in the same folder as the main program.

Please click to download the main program(and the expert UI is included).
Modify the button event code.

Open the main program. And all the UI file in the same folder will open too.

First, define a variable in the ui_Screen1.c file to store the status of the LED. In the main program, the LED light is controlled to turn on and off by judging the status of this variable.

Then replace the function calls inside the Button1 and Button2 functions with the functions for turning the LED on and turning the LED off

The UI file export is completed, and the button event function is also modified. Next we're going to learn about the Arduino main program and learn how to upload the code to the board.

Build the Project with Arduino IDE¶

Get Started with Arduino IDE¶

Please click the card below to learn how to install Arduino IDE, and install ESP32 board in the Arduino IDE.

Install Libraries¶

In this project, we will use the following libraries:

#include <lvgl.h>
#include <SPI.h>
#include <Arduino_GFX_Library.h>
#include "ui.h"
#include "touch.h"

#include <lvgl.h> : library for graphical interfaces.
#include <Arduino_GFX_Library.h>: library for screen drive
touch.h: touch drive.
#include "ui.h": is a custom header file that contain UI-related functions or definitions.

Please click to download the libraries we modified.

Then click "File" -> "Preferences" -> "Setting" to check the sketchbook location. Place the libraries downloaded to the sketchbook location.

Code Explanation¶

Basic Definition

Backlight

#define TFT_BL 2
#define GFX_BL DF_GFX_BL // default backlight pin, you may replace DF_GFX_BL to actual backlight pin

Create a default Arduino_GFX object lcd

#if defined(DISPLAY_DEV_KIT)
Arduino_GFX *lcd = create_default_Arduino_GFX();
#else

int led;
SPIClass& spi = SPI;

// ========== NV3047 480x272 timing parameters ==========
// DCLK: typical 9MHz (range 8-12MHz)
// timing: hsync_fp=8, hsync_pw=4, hsync_bp=43
//       vsync_fp=8, vsync_pw=4, vsync_bp=12

Arduino_ESP32RGBPanel *bus = new Arduino_ESP32RGBPanel(
  // RGB control pins
  40 /* DE */, 41 /* VSYNC */, 39 /* HSYNC */, 42 /* PCLK */,

  // RGB data pins
  45 /* R0 */, 48 /* R1 */, 47 /* R2 */, 21 /* R3 */, 14 /* R4 */,
  5  /* G0 */, 6  /* G1 */, 7  /* G2 */, 15 /* G3 */, 16 /* G4 */, 4 /* G5 */,
  8  /* B0 */, 3  /* B1 */, 46 /* B2 */, 9  /* B3 */, 1  /* B4 */,

  // timing parameters（consistent with NV3047 datasheet）
  0  /* hsync_polarity */, 8  /* hsync_front_porch */, 4  /* hsync_pulse_width */, 43 /* hsync_back_porch */,
  0  /* vsync_polarity */, 8  /* vsync_front_porch */, 4  /* vsync_pulse_width */, 12 /* vsync_back_porch */,

  // PCLK settings
  1  /* pclk_active_neg */, 9000000 /* prefer_speed: 9MHz（NV3047 typical value）*/, false /* useBigEndian */,
  0  /* de_idle_high */, 0  /* pclk_idle_high */, 480 /* bounce_buffer_size_px */
);

Arduino_RGB_Display *lcd = new Arduino_RGB_Display(
  480 /* width */, 272 /* height */, bus, 0 /* rotation */, false /* auto_flush */
);
/*It uses a class called Arduino_ESP32RGBPanel to initialize a display bus object bus. This object is used to control the ESP32-based RGB display panel. Then, according to option 1 in the comments, it uses the Arduino_RPi_DPI_RGBPanel class to initialize a display object called lcd. This object is used to control an ILI6485 LCD with a resolution of 480x272 pixels.*/

Resolution

/*Changing the screen resolution*/
static uint32_t screenWidth;
static uint32_t screenHeight;

Define a buffer variable to initialize the display driver of LVGL and prepare a buffer to store pixel data that will be rendered to the screen. The size of this buffer should be around 1/10 of the product of screen pixel values, which can balance the main control performance and the smoothness of screen display. Taking 10 as an example, the display will be smoother.
```
static lv_disp_draw_buf_t draw_buf;
static lv_color_t disp_draw_buf[480 * 272 / 8];
static lv_disp_drv_t disp_drv;
```
Define the led variable and the instance of the SPI library
```
int led;
SPIClass& spi = SPI;
```

Function

The my_disp_flush() function is used to refresh the display. It receives a display driver pointer disp of type lv_disp_drv_t, a lv_area_t structure pointer area representing the refresh area, and a lv_color_t pointer color_p pointing to the color data. In the function, the width w and height h of the refresh area are first calculated. Then, depending on whether the LV_COLOR_16_SWAP macro is defined, different methods are selected to draw the color data to the display device. If the LV_COLOR_16_SWAP macro is defined, the draw16bitBeRGBBitmap() method is used to draw the color data; otherwise, the draw16bitRGBBitmap() method is used. Finally, the lv_disp_flush_ready() function is called to notify LittlevGL that the display has been refreshed.

/* Display Refresh */
/* Display flushing */
void my_disp_flush(lv_disp_drv_t *disp, const lv_area_t *area, lv_color_t *color_p)
{
  uint32_t w = (area->x2 - area->x1 + 1);
  uint32_t h = (area->y2 - area->y1 + 1);

#if (LV_COLOR_16_SWAP != 0)
  lcd->draw16bitBeRGBBitmap(area->x1, area->y1, (uint16_t *)&color_p->full, w, h);
#else
  lcd->draw16bitRGBBitmap(area->x1, area->y1, (uint16_t *)&color_p->full, w, h);
#endif

  lv_disp_flush_ready(disp);
}

The my_touchpad_read() function is used to process touch input. It receives an input device driver pointer indev_driver of type lv_indev_drv_t, and a pointer to the lv_indev_data_t structure data representing the input data. In the function, first check whether the touch screen has a signal. If there is a signal, further check whether the touch is pressed or released. If the touch is pressed, set the input state to LV_INDEV_STATE_PR, and set the coordinates of the input point to the coordinates of the last touch of the touch screen. If the touch is released, set the input state to LV_INDEV_STATE_REL. If there is no signal on the touch screen, set the input state to LV_INDEV_STATE_REL, indicating that the input device is in the released state.

void my_touchpad_read(lv_indev_drv_t *indev_driver, lv_indev_data_t *data)
{
  if (touch_has_signal())
  {
    if (touch_touched())
    {
      data->state = LV_INDEV_STATE_PR;
      data->point.x = touch_last_x;
      data->point.y = touch_last_y;
      Serial.print("Data x :");
      Serial.println(touch_last_x);
      Serial.print("Data y :");
      Serial.println(touch_last_y);
    }
    else if (touch_released())
    {
      data->state = LV_INDEV_STATE_REL;
    }
  }
  else
  {
    data->state = LV_INDEV_STATE_REL;
  }
}

Setup Part

void setup()
{
  Serial.begin(115200);
  delay(1000);
  Serial.println("Starting...");

  pinMode(LED, OUTPUT);
  digitalWrite(LED, LOW);
  pinMode(0, OUTPUT); // TOUCH-CS

  lv_init();

#ifdef TFT_BL
  pinMode(TFT_BL, OUTPUT);
  digitalWrite(TFT_BL, HIGH);
  Serial.println("Backlight ON");
#endif

  Serial.println("Init display...");
  if (!lcd->begin())
  {
    Serial.println("lcd->begin() FAILED!");
    while (1) delay(100);
  }
  Serial.println("lcd->begin() OK");

  lcd->setTextSize(2);
  lcd->fillScreen(0);

  touch_init();

        /*Screen display initialization, by calling the lv_disp_draw_buf_init function to initialize the drawing buffer. This function accepts three parameters: &draw_buf is a pointer to the lv_disp_draw_buf_t structure, buf1 is a buffer for storing pixel data, and the last parameter is NULL to indicate that no additional buffer is used. The buffer size calculated is screenWidth * screenHeight / 8, and it is associated with the drawing buffer structure.
The size of this buffer affects the smoothness. The larger the buffer, the smoother the display. The size is limited by its own memory.*/

  screenWidth = lcd->width();
  screenHeight = lcd->height();
  Serial.print("Screen: ");
  Serial.print(screenWidth);
  Serial.print("x");
  Serial.println(screenHeight);

  lv_disp_draw_buf_init(&draw_buf, disp_draw_buf, NULL, screenWidth * screenHeight / 8);

  lv_disp_drv_init(&disp_drv);
  disp_drv.hor_res = screenWidth;
  disp_drv.ver_res = screenHeight;
  disp_drv.flush_cb = my_disp_flush;
  disp_drv.draw_buf = &draw_buf;
  lv_disp_drv_register(&disp_drv);

        /*Initialize the (virtual) input device driver. This code is used to initialize a virtual input device driver and associate it with the LVGL framework to handle the interaction of the input device. It provides the necessary configuration and functions for the input device driver by setting the input device type and defining the read callback function.*/
     /* Initialize the (dummy) input device driver */

  static lv_indev_drv_t indev_drv;
  lv_indev_drv_init(&indev_drv);
  indev_drv.type = LV_INDEV_TYPE_POINTER;
  indev_drv.read_cb = my_touchpad_read;
  lv_indev_drv_register(&indev_drv);

  ui_init();
  lv_timer_handler();

  Serial.println("Setup done");
}

Main Program Part

void loop()
{
  lv_timer_handler();

  if(led == 1)
    digitalWrite(LED, HIGH);
  if(led == 0)
    digitalWrite(LED, LOW);

  delay(16);  // approx 60fps，matching NV3047 typical frame rate
}

Upload the Code¶

After completing the installation of the ESP32 board according to "Get Started with Arduino IDE" and the installation of the library according to "Install Libraries", open the program and connect the ESP32 HMI 4.3-inch to the computer via a USB-C cable.
Select Board: click "Tools" -> "Board" -> "esp32" and select "ESP32S3 Dev Module"
Board Setting: Under the "Tools" menu, see "Flash Size" select 4MB(32Mb); "Partition scheme" and select "Huge APP(3MB No OTA/1MB SPIFFS)"; "PSRAM" select "QSPI PSRAM"
Select Port
Upload the code: click the "Upload" icon to upload the code.
After the program is successfully uploaded, connect the LED to GPIO_D. Click the ON button on the screen and the LED will turn on. Click the OFF button and the LED will turn off.

Example Demo of ESP32 HMI Function¶

Note:

The library used in this section is in the compressed package of the library downloaded above.

Example1: LED blinking.¶

Connect the LED to GPIO_D(IO38) port, and upload the following code to the board. The LED will blinking.

#define D_PIN 38

void setup() {
  // put your setup code here, to run once:
  Serial.begin(115200);
  pinMode(D_PIN, OUTPUT);
}

void loop() {
  // put your main code here, to run repeatedly:
  digitalWrite(D_PIN, HIGH);
  delay(500);
  digitalWrite(D_PIN, LOW);
  delay(500);
}

Example2: Play music¶

Plug the speaker into SPK port. Please check if you have installed the library ESP32-audioI2S(Copy the ESP32-audioI2S-master folder in the libraries downloaded above to the .../arduino/libraries directory). Upload the following code to the board.

#include <Wire.h>
#include <SPI.h>
#include <SD.h>
#include <FS.h>
#include "Audio.h"
Audio audio;

#define I2S_DOUT      20
#define I2S_BCLK      35
#define I2S_LRC       19

//4.3
#define SD_MOSI 11
#define SD_MISO 13
#define SD_SCK 12
#define SD_CS 10

void setup() {
  // put your setup code here, to run once:
  Serial.begin( 9600 ); 
  pinMode(SD_CS, OUTPUT);      digitalWrite(SD_CS, HIGH);
  SPI.begin(SD_SCK, SD_MISO, SD_MOSI);
  SPI.setFrequency(1000000);
  SD.begin(SD_CS);
  audio.setPinout(I2S_BCLK, I2S_LRC, I2S_DOUT);
  audio.setVolume(21); // 0...21
  audio.connecttoFS(SD, "/123.mp3");
}

void loop() {
  // put your main code here, to run repeatedly:
  audio.loop();
  //audio.stopSong();//CLOSE
}

Example3: Initialize SD Card slot¶

Please insert an SD card formatted as FAT16 or FAT32. Other file system formats may fail to be read.

#include <Wire.h>
#include <SPI.h>
#include <SD.h>
#include <FS.h>

//4.3
#define SD_MOSI 11
#define SD_MISO 13
#define SD_SCK 12
#define SD_CS 10
void setup() {
  // put your setup code here, to run once:
  Serial.begin(115200); 
  SPI.begin(SD_SCK, SD_MISO, SD_MOSI);
  delay(100);
  if (SD_init() == 1)
  {
    Serial.println("Card Mount Failed");
  }
  else
    Serial.println("Initialize SD Card successfully");
}

void loop() {
  // put your main code here, to run repeatedly:

}

//SD Card initialization
int SD_init()
{

  if (!SD.begin(SD_CS))
  {
    Serial.println("Card Mount Failed");
    return 1;
  }
  uint8_t cardType = SD.cardType();

  if (cardType == CARD_NONE)
  {
    Serial.println("No TF card attached");
    return 1;
  }

  uint64_t cardSize = SD.cardSize() / (1024 * 1024);
  Serial.printf("TF Card Size: %lluMB\n", cardSize);
  listDir(SD, "/", 2);


  return 0;
}

//Traverse the SD card directory
void listDir(fs::FS & fs, const char *dirname, uint8_t levels)
{
  //  Serial.printf("Listing directory: %s\n", dirname);

  File root = fs.open(dirname);
  if (!root)
  {
    //Serial.println("Failed to open directory");
    return;
  }
  if (!root.isDirectory())
  {
    Serial.println("Not a directory");
    return;
  }

  File file = root.openNextFile();
//  i = 0;
  while (file)
  {
    if (file.isDirectory())
    {

      if (levels)
      {
        listDir(fs, file.name(), levels - 1);
      }
    }
    else
    {
      Serial.print("FILE: ");
      Serial.print(file.name());

      Serial.print("SIZE: ");
      Serial.println(file.size());
    }

    file = root.openNextFile();
  }
}

Example4: Initialize the touch¶

Upload the following code to the board, and open the serial monitor to check the touch information.

#include "touch.h"

void setup() {
  // put your setup code here, to run once:
  Serial.begin( 115200 ); 
  touch_init();
}

void loop() {
  // put your main code here, to run repeatedly:
  if (touch_has_signal())
  {
    if (touch_touched())
    {
      Serial.print( "Data x :" );
      Serial.println( touch_last_x );

      Serial.print( "Data y :" );
      Serial.println( touch_last_y );
    }
  }
}

Example5: BLE¶

Upload the following code to the board, and use the phone to search the Bluetooth device.

#include "BLEDevice.h"              //BLE drive library
#include "BLEServer.h"              //BLE server library
#include "BLEUtils.h"               //BLE Utility library
#include "BLE2902.h"                //Feature Add Descriptor Library
#include <BLECharacteristic.h>      //BLE Feature function library
BLEAdvertising* pAdvertising = NULL;
BLEServer* pServer = NULL;
BLEService *pService = NULL;
BLECharacteristic* pCharacteristic = NULL;
#define bleServerName "ESP32SPI-BLE"  //BLE Server name
#define SERVICE_UUID "6479571c-2e6d-4b34-abe9-c35116712345"  //Server UUID
#define CHARACTERISTIC_UUID "826f072d-f87c-4ae6-a416-6ffdcaa02d73"


bool connected_state = false;   //Create device connection identifier

class MyServerCallbacks: public BLEServerCallbacks  //Create connection and disconnection calling classes
{
    void onConnect(BLEServer *pServer)//Start connecting function
    {
      connected_state = true;   //Device is connected correctly
    }
    void onDisconnect(BLEServer *pServer)//disconnect function
    {
      connected_state = false;  //Device connection error
    }

};
void setup() {
  // put your setup code here, to run once:
  Serial.begin(115200);
  //BLE
  BLEDevice::init(bleServerName);  //Create BLE and set name
  pServer = BLEDevice::createServer();  //Create BLE server
  pServer->setCallbacks(new MyServerCallbacks());  //Set up connection and disconnection calling classes
  pService = pServer->createService(SERVICE_UUID); //Create BLE service

  pCharacteristic = pService->createCharacteristic(  //Create ble feature（Characterristic_UUID）
                      CHARACTERISTIC_UUID,
                      BLECharacteristic::PROPERTY_READ | BLECharacteristic::PROPERTY_WRITE | BLECharacteristic::PROPERTY_NOTIFY);
  pCharacteristic->setValue("ELECROW");
  //start broadcast
  pAdvertising = BLEDevice::getAdvertising();  //A bleadvertising class pointer padvertising is defined, which points to bledevice:: getadvertising()
  pAdvertising->addServiceUUID(SERVICE_UUID);
  pAdvertising->start();  //Start broadcasting
  pService->start();
  //  pAdvertising->stop();  //Stop broadcasting
  //  pService->stop();      //Stop service
}

void loop() {

}

Example6: Initialize the WIFI¶

Upload the following code to ESP display，Note: Please change the Wi-Fi SSID and password to your own.

#include <WiFi.h>

const char *ssid = "elecrow888"; //Change to your SSID
const char *password = "elecrow2014"; //Change to your password
void setup() {
  // put your setup code here, to run once:
  Serial.begin(115200);
  WiFi.begin(ssid, password);
  WiFi.setAutoReconnect(true);//Set up automatic reconnection
  while (WiFi.status() != WL_CONNECTED) {
    delay(100);
    Serial.println("connecting");
  }
  Serial.println("WiFi is connected.");
  Serial.println("IP address: ");
  Serial.println(WiFi.localIP());
//  WiFi.disconnect();
}

void loop() {

}

Example7: Connect Crowtail-GPS module via UART to Get Location¶

Connect the GPS module to the UART interface. After running the program, you can view the returned GPS data in the serial monitor. It is best to be outdoors for better GPS signal reception.

GPS module purchase link:https://www.elecrow.com/crowtailgps-p-1515.html

#define SERIAL_BAUD 9600
HardwareSerial cardSerial(1);//Declare the UART1
unsigned char buffer[256]; // buffer array for data recieve over serial port
int count_1 = 0;   // counter for buffer array
void clearBufferArray()              // function to clear buffer array
{
  for (int i = 0; i < count_1; i++)
  {
    buffer[i] = 0; // clear all index of array with command NULL
  }
}

void setup() {
  // put your setup code here, to run once:
  Serial.begin( 115200 ); /*Initialize the serial port*/
  cardSerial.begin(SERIAL_BAUD, SERIAL_8N1, 18, 17);//4.3 /*初Initialize the serial port1*/18-RX\17-TX
}

void loop() {
  // put your main code here, to run repeatedly:
  if (cardSerial.available())              // if date is comming from softwareserial port ==> data is comming from SoftSerial shield
  {
    while (cardSerial.available())         // reading data into char array
    {
      buffer[count_1++] = cardSerial.read();   // writing data into array
      if (count_1 == 256)break;
    }
    Serial.write(buffer, count_1);           // if no data transmission ends, write buffer to hardware serial port
    clearBufferArray();              // call clearBufferArray function to clear the storaged data from the array
    count_1 = 0;                       // set counter of while loop to zero
  }
  if (Serial.available())            // if data is available on hardwareserial port ==> data is comming from PC or notebook
    cardSerial.write(Serial.read());       // write it to the SoftSerial shield
}