Issue loading files w/SPIFFS (ERR_CONTENT_LENGTH_MISMATCH)

[Th3On3Fr33man]

Basic Info

Hardware

Hardware: ESP-8266
Core Version: 2.2.0

Description

Alright so I've been looking into this for the past two days and I still feel like I'm getting nowhere.

I recently started using the SPIFFS File System for Arduino development on an HUZZAH ESP8266 like the FSBrowser.ino example, and while it's been great in terms of separating code, as my code continues to grow it has not been great in terms of stability.

Ever since I began adding more and more javascript, I began to have errors pop up for various files, whether it's my HTML/CSS/JS, and the primary error I see is ERR_CONTENT_LENGTH_MISMATCH.

//File Read for File System
bool handleFileRead(String path)
{
  if(mySerial)  
    mySerial.println("handleFileRead: " + path);
  if(path.endsWith("/")) path += "index.htm";
  String contentType = getContentType(path);
  String pathWithGz = path + ".gz";
  if(SPIFFS.exists(pathWithGz) || SPIFFS.exists(path))
  {


    if(SPIFFS.exists(pathWithGz))
      path += ".gz";
    File file = SPIFFS.open(path, "r");

    if(mySerial)
        mySerial.println("TEST: " + path + " FILE OPEN: " + file.size());

    size_t sent = server.streamFile(file, contentType);

    if(mySerial)
        mySerial.println("TEST: " + path + " SIZE: " + file.size());

    file.close();

    if(mySerial)
        mySerial.println("TEST: " + path + " FILE CLOSE");

    return true;
  } //end if 
  return false;
} //end function handleFileRead

So after that happened I began examining the handleFileRead() function that I copied from the FSBrowser.h example and after testing various print statements it seems the hangup occurs within this line:

size_t sent = server.streamFile(file, contentType);

template<typename T> size_t streamFile(T &file, const String& contentType){
  setContentLength(file.size());
  if (String(file.name()).endsWith(".gz") &&
      contentType != "application/x-gzip" &&
      contentType != "application/octet-stream"){
    sendHeader("Content-Encoding", "gzip");
  }
  send(200, contentType, "");
  return _currentClient.write(file, HTTP_DOWNLOAD_UNIT_SIZE);
}

So from there, I went into ESP8266WebServer.h and found the streamFile() function. Once inside this function, I discovered this line:

return _currentClient.write(file, HTTP_DOWNLOAD_UNIT_SIZE);

From this I deduced two things:

• There is a download limitation capping at HTTP_DOWNLOAD_UNIT_SIZE (which is defined at 1460).
• This function calls another function within the WiFiClient class called write().

So I followed the trail to this function: size_t WiFiClient::write(const uint8_t *buf, size_t size).

size_t WiFiClient::write(const uint8_t *buf, size_t size)
{

    SoftwareSerial mySerial(12,13); 

    mySerial.begin(115200);

    if(mySerial)
    {
        mySerial.print("Size: ");
        mySerial.println(size);
    }

    if (!_client || !size)
    {
        if(mySerial)
            mySerial.println("FAILURE");

        return 0;
    }

        if(mySerial)
            mySerial.println("SUCCESS");

    return _client->write(reinterpret_cast<const char*>(buf), size);
}

Now this doesn't quite match because the first parameter of the write() function that was called is of type File, while the first parameter of this function is uint8_t, but I placed some print statements within here nonetheless and they execute as the files are being loaded so I believe I've found the correct function.

However, now I'm at a loss as to how to continue fixing the problem from here, since there is only a recursive call and the error seems to occur because the culprit file doesn't finish loading (at least according to chrome debugger's network tab).

As you can see, the style.css file is listed as being 3 kilobytes, which is of course wrong: my css file is 27 kilobytes.

So it appears that there's an issue loading files (this is not exclusive to the css file, it has happened with my javascript files as well and appears to be random). I have tried minifying all the files and it hasn't fixed the issue; so I could really use some help determining how to fix this.

Now I received some advice from another source to try using this library, which is a modified version of the FSBrowser code: https://github.com/me-no-dev/ESPAsyncWebServer

At first this seemingly had fixed all my problems. The code was running stable with none of the previous issues popping up. But since switching to that library, I've run into a myriad of new issues:

And the strangest thing happened, where it appears some of the code was missing?

If you look at the red highlighted line in that photo, you can see that in the middle of the removeClass() statement, it stops spelling out the class (which is "deactiveSect") and starts a completely different if statement further down in the code.

And yet here's the same portion of code on the server currently:

You can see it seemingly merged lines 66 and 70.

So while the version of the FSBrowser code that I was using couldn't handle the level of code I was working with, I'm failing to see how I can afford to use this modified library if this myriad of issues keeps occurring.

Settings in IDE

Module: Adafruit Huzzah ESP8266
Flash Size: 4MB/3MB
CPU Frequency: 80Mhz
Flash Mode: ??? (Unsure where to find this)
Flash Frequency: ??? (Unsure where to find this)
Upload Using: ??? (Unsure where to find this...I'm using a USB to FTDI cable so...Serial?)
Reset Method: ??? (Unsure where to find this...I'm using the reset button on the card itself?)
Upload speed: 115200
Port: COM8
Programmer: AVRISP mkII

Here's the source for the rest of my code (with the original FSBrowser library):

Primary ino file

//Header File
#include "HHIO_Portal.h"

//Function used to verify incoming information from the teensy to the wifi card via the communication protocol
void recInfoWifi()
{
    /*
    ===============================
    COMMUNICATION PROTOCOL
    ===============================
    */
    if(Serial.available() > 0)
    {
        //SOM
        if(initialCounter == 0)
        {
            rx_byte = Serial.read();

            //Print Start of Message
            //if(mySerial)
                //mySerial.print("0x");
            if(rx_byte<0x10)
            {
                //if(mySerial)
                    //mySerial.print("0");
            } //end if

            //if(mySerial)
                //mySerial.println(rx_byte, HEX);

            //Finish the SOM reading sequence
            initialCounter++;
        } //end if

        //Invalid Message
        if((!messageBegun) && (rx_byte != som) && (invalidMessageFlag == 0))
            {
                invalidMessageFlag = 1;

                //if(mySerial)
                    //mySerial.println("INVALID MESSAGE START");
            } //end if

        //If the SOM was read successfully...
        if((!messageBegun) && (rx_byte == som) && (invalidMessageFlag == 0))
        {
            messageBegun = true;

            //if(mySerial)
                //mySerial.println("MESSAGE BEGUN TRUE");
        } //end if (messageInProgress && rx_byte)

        //Message Length
        if((messageBegun) && (!messageInProgress) && (invalidMessageFlag == 0))
        {
            serialCurrent = Serial.available();

            if(serialCurrent > 0)
            {      
                receivedMessageLength = (uint8_t)Serial.read();

                //if(mySerial)
                //{
                    //mySerial.print("MESSAGE LENGTH: ");
                    //mySerial.println(receivedMessageLength);
                //} //end if



                messageInProgress = true;

                //if(mySerial)
                    //mySerial.println("MESSAGE IN PROGRESS TRUE");

            } //end if (serialCurrent)
        } //end if (messageBegun && messageInProgress)

        //Subsystem, Message Content and EOM
        if((messageInProgress) && (invalidMessageFlag == 0))
        {
            serialCurrent = Serial.available(); 

            if(serialCurrent >= receivedMessageLength)
            {

                Serial.readBytes(rxBuff, receivedMessageLength);

                if((byte)rxBuff[receivedMessageLength-1] != eom)
                {
                    //if(mySerial)
                    //{
                        //mySerial.println("ERROR");
                        //mySerial.write(Serial.read());
                    //} //end if

                } //end if (rxBuff != eom)

                else
                { 
                    /*
                    ===============================
                    SUBSYSTEM
                    ===============================
                    */
                    receivedSubsys = rxBuff[0]; 

                        //Print Subsystem
                        //if(mySerial)
                            //mySerial.print("0x");
                        if(receivedSubsys<0x10)
                        {
                            //if(mySerial)
                                //mySerial.print("0");
                        } //end if

                        //if(mySerial)
                            //mySerial.println(receivedSubsys, HEX);

                    /*
                    ===============================
                    MESSAGE CONTENT
                    ===============================
                    */

                    //NUM REFRESH
                    if(receivedSubsys == SUBSYS_14)
                    {
                        currNumRefresh = rxBuff[1];

                        //END OF MESSAGE
                        //if(mySerial)
                        //{
                            //mySerial.print("0x");
                            //mySerial.println(rxBuff[2], HEX);
                        //} //end if


                        initialCounter = 0;
                        messageBegun = false;
                        messageInProgress = false;
                        receivedMessageLength = 0;

                    } //end if (SUBSYS_14)

                    //MICROSWITCH C
                    else if(receivedSubsys == SUBSYS_4)
                    {
                        currMicroC = rxBuff[1];

                        //END OF MESSAGE
                        //if(mySerial)
                        //{
                            //mySerial.print("0x");
                            //mySerial.println(rxBuff[2], HEX);
                        //} //end if

                        initialCounter = 0;
                        messageBegun = false;
                        messageInProgress = false;
                        receivedMessageLength = 0;

                    } //end else if (SUBSYS_4)

                    else if(receivedSubsys == SUBSYS_5)
                    {
                        currMicroD = rxBuff[1];

                        //END OF MESSAGE
                        //if(mySerial)
                        //{
                            //mySerial.print("0x");
                            //mySerial.println(rxBuff[2], HEX);
                        //} //end if

                        initialCounter = 0;
                        messageBegun = false;
                        messageInProgress = false;
                        receivedMessageLength = 0;

                    } //end else if (SUBSYS_5)

                    else if(receivedSubsys == SUBSYS_6)
                    {
                        currMicroE = rxBuff[1];

                        //END OF MESSAGE
                        //if(mySerial)
                        //{
                            //mySerial.print("0x");
                            //mySerial.println(rxBuff[2], HEX);
                        //} //end if

                        initialCounter = 0;
                        messageBegun = false;
                        messageInProgress = false;
                        receivedMessageLength = 0;

                    } //end else if (SUBSYS_6)

                    else if(receivedSubsys == SUBSYS_7)
                    {
                        currPressureC = rxBuff[1];

                        //END OF MESSAGE
                        //if(mySerial)
                        //{
                            //mySerial.print("0x");
                            //mySerial.println(rxBuff[2], HEX);
                        //} //end if

                        initialCounter = 0;
                        messageBegun = false;
                        messageInProgress = false;
                        receivedMessageLength = 0;

                    } //end else if (SUBSYS_7)

                    else if(receivedSubsys == SUBSYS_8)
                    {
                        currPressureD = rxBuff[1];

                        //END OF MESSAGE
                        //if(mySerial)
                        //{
                            //mySerial.print("0x");
                            //mySerial.println(rxBuff[2], HEX);
                        //} //end if

                        initialCounter = 0;
                        messageBegun = false;
                        messageInProgress = false;
                        receivedMessageLength = 0;

                    } //end else if (SUBSYS_8)

                    else if(receivedSubsys == SUBSYS_9)
                    {
                        currPressureE = rxBuff[1];

                        //END OF MESSAGE
                        //if(mySerial)
                        //{
                            //mySerial.print("0x");
                            //mySerial.println(rxBuff[2], HEX);
                        //} //end if

                        initialCounter = 0;
                        messageBegun = false;
                        messageInProgress = false;
                        receivedMessageLength = 0;

                    } //end else if (SUBSYS_9)

                    else if(receivedSubsys == SUBSYS_10)
                    {
                        currValveStatusNumC = rxBuff[1];

                        //END OF MESSAGE
                        //if(mySerial)
                        //{
                            //mySerial.print("0x");
                            //mySerial.println(rxBuff[2], HEX);
                        //} //end if

                        initialCounter = 0;
                        messageBegun = false;
                        messageInProgress = false;
                        receivedMessageLength = 0;

                    } //end else if (SUBSYS_10)

                    else if(receivedSubsys == SUBSYS_11)
                    {
                        currValveStatusNumD = rxBuff[1];

                        //END OF MESSAGE
                        //if(mySerial)
                        //{
                            //mySerial.print("0x");
                            //mySerial.println(rxBuff[2], HEX);
                        //} //end if

                        initialCounter = 0;
                        messageBegun = false;
                        messageInProgress = false;
                        receivedMessageLength = 0;

                    } //end else if (SUBSYS_11)

                    else if(receivedSubsys == SUBSYS_12)
                    {
                        currValveStatusNumE = rxBuff[1];

                        //END OF MESSAGE
                        //if(mySerial)
                        //{
                            //mySerial.print("0x");
                            //mySerial.println(rxBuff[2], HEX);
                        //} //end if

                        initialCounter = 0;
                        messageBegun = false;
                        messageInProgress = false;
                        receivedMessageLength = 0;

                    } //end else if (SUBSYS_12)

                    else if(receivedSubsys == SUBSYS_13)
                    {
                        currFluid = rxBuff[1];

                        //END OF MESSAGE
                        //if(mySerial)
                        //{
                            //mySerial.print("0x");
                            //mySerial.println(rxBuff[2], HEX);
                        //} //end if

                        initialCounter = 0;
                        messageBegun = false;
                        messageInProgress = false;
                        receivedMessageLength = 0;

                    } //end else if (SUBSYS_13)
                } //end else
            } //end if (serialCurrent)
        } //end if (messageInProgress)
    } //end if (Serial.available)
} //end comProtocolWifi

//Function used to send information to the teensy in order to control the HHIO PTT
void sendInfoWifi()
{
    unsigned long currTime = millis();

    //SEND
    if(currTime - prevTimeTest > testInterval)   
    {
        prevTimeTest = currTime; 

        //Test Message
        messageLength = 3;
        subsys = SUBSYS_DBG;

        messageContent = wifiCounter; 

        //Teensy Console
        Serial.write(som);
        Serial.write(messageLength);
        Serial.write(subsys);
        Serial.write(messageContent);
        Serial.write(eom);

        wifiCounter++;

    } //end if (RECEIVE INFO)
} //end function sendInfoWifi

void setup()
{ 
  //Initiate Serial connection to Teensy
  Serial.begin(115200);

  //This is for debugging/housekeeping (And all console messages that use mySerial as opposed to Serial)
  mySerial.begin(115200);

  /*
  ===============================
  FILE SYSTEM
  ===============================
  */
  SPIFFS.begin();
  {
    Dir dir = SPIFFS.openDir("/");
    while (dir.next()) {    
      String fileName = dir.fileName();
      size_t fileSize = dir.fileSize();
      if(mySerial)
        mySerial.printf("FS File: %s, size: %s\n", fileName.c_str(), formatBytes(fileSize).c_str()); 
    } //end while
        if(mySerial)
            mySerial.printf("\n"); 
  } //end SPIFFS BEGIN  

  /*
  ===============================
  WIFI INIT
  ===============================
  */
  if(mySerial)
    mySerial.printf("Connecting to %s\n", ssid); 

  //Make the initial Wifi connection
  if (String(WiFi.SSID()) != String(ssid)) {
    WiFi.begin(ssid, password);
  } //end if

 //Wait for the Wifi to finish connecting
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    if(mySerial)
      mySerial.print("."); 
  } //end while

  //IP Address for webpage
  if(mySerial)
  {
      mySerial.println(""); 
      mySerial.print("Connected! IP address: "); 
      mySerial.println(WiFi.localIP()); 
  } //end if

  //Might not need this
  MDNS.begin(host);

  //Hostname for webpage
  if(mySerial)
  {
    mySerial.print("Open http://"); 
    mySerial.print(host); 
    mySerial.println(".local/edit to see the file browser"); 
  } //end if


  /*
  ===============================
  SERVER INIT
  ===============================
  */
  //list directory
  server.on("/list", HTTP_GET, handleFileList);
  //load editor
  server.on("/edit", HTTP_GET, [](){
    if(!handleFileRead("/edit.htm")) 
        server.send(404, "text/plain", "FileNotFound");
  });
  //create file
  server.on("/edit", HTTP_PUT, handleFileCreate);
  //delete file
  server.on("/edit", HTTP_DELETE, handleFileDelete);
  //first callback is called after the request has ended with all parsed arguments
  //second callback handles file uploads at that location
  server.on("/edit", HTTP_POST, [](){ server.send(200, "text/plain", ""); }, handleFileUpload);

  //called when the url is not defined here
  //use it to load content from SPIFFS
  server.onNotFound([](){
    if(!handleFileRead(server.uri()))
      server.send(404, "text/plain", "FileNotFound");
  });

  //get heap status, analog input value and all GPIO statuses in one json call
  server.on("/all", HTTP_GET, [](){
    String json = "{";
    json += "\"heap\":"+String(ESP.getFreeHeap());
    json += ", \"analog\":"+String(analogRead(A0));
    json += ", \"gpio\":"+String((uint32_t)(((GPI | GPO) & 0xFFFF) | ((GP16I & 0x01) << 16)));
    json += "}";
    server.send(200, "text/json", json);
    json = String();
  });

  //Start the server
  server.begin();

  if(mySerial)  
    mySerial.println("HTTP server started"); 

  //Open the websocket connection in order to update the status values on the page without refreshing
  webSocket.begin();
  webSocket.onEvent(webSocketEvent);
} //end Setup

void loop()
{   
    //SEND INFO TO TEENSY
    sendInfoWifi();

    //RECEIVE INFO FROM TEENSY
    recInfoWifi();

    //CONTINUE WEBSOCKET CONNECTION
    webSocket.loop();

    //SERVE WEBPAGE TO THE USER
    server.handleClient();
} //end loop

Primary ino header file

//Initiate necessary libraries
#include <SoftwareSerial.h>
#include <HHIO_Subsystems.h>
#include <ESP8266WiFi.h>
//#include <pfodESP8266WiFi.h>
#include <WebSocketsServer.h>
//#include <pfodESP8266WebServer.h>
#include <ESP8266WebServer.h>
#include <ESP8266mDNS.h>
#include <FS.h>

//Serial connection for debugging (RX, TX)
SoftwareSerial mySerial(12,13); 

//SSID/Pass/Host
const char* ssid = "*****";
const char* password = "*****";
const char* host = "esp8266fs";

//Setup File System Connection
MDNSResponder mdns;

//Web Server
ESP8266WebServer server(80);

//File System 
File fsUploadFile;

//Websocket
WebSocketsServer webSocket = WebSocketsServer(81);

//Invalid Message
uint8_t invalidMessageFlag = 0;

//Counter for testing serial connection
uint8_t wifiCounter = 0;

//Com. Protocol Buffer
uint8_t rxBuff[256];

//Start of Message
uint8_t som = 0x11;

//End of Message
uint8_t eom = 0x12;

//Subsystem variables
uint8_t subsys; 
uint8_t receivedSubsys;

//Read Byte for Com. Protocol
uint8_t rx_byte = 0x00;

//Message length variables
uint8_t messageLength = 0;
uint8_t receivedMessageLength = 0;

//Message content variable
uint8_t messageContent;

//Variable to check incoming serial content
uint8_t serialCurrent; 

//Counter used for verifying the som variable
int initialCounter = 0;

//Message progression variables
boolean messageBegun = false; 
boolean messageInProgress = false;

//Variables currently used to read status info coming from the teensy (CURRENTLY RANDOMIZED INFO)
int currNumRefresh = 0;
int currMicroC = 0;
int currMicroD = 0;
int currMicroE = 0;
int currPressureC = 0; 
int currPressureD = 0; 
int currPressureE = 0; 
int currValveStatusNumC = 0; 
int currValveStatusNumD = 0; 
int currValveStatusNumE = 0;
int currFluid = 0;

//Buffer for sending information from Arduino to the webpage
char statusbuf[256]; 

//Valve 1
String valveStatusC = "Closed";
String valveTubePropsC = "175px solid #00ADEF";
String valveColorC = "red";

//Valve 2
String valveStatusD = "Closed";
String valveTubePropsD = "175px solid #00ADEF";
String valveColorD = "red";

//Valve 3
String valveStatusE = "Closed";
String valveTubePropsE = "175px solid #00ADEF";
String valveColorE = "red";

//Variables for sending test message to teensy
long prevTimeTest = 0;
long testInterval = 5000;

//Format Bytes
String formatBytes(size_t bytes){

  if(mySerial)  
    mySerial.println("formatBytes FUNC");

  if (bytes < 1024){
    return String(bytes)+"B";
  } //end if 

  else if(bytes < (1024 * 1024)){
    return String(bytes/1024.0)+"KB";
  } //end else if

  else if(bytes < (1024 * 1024 * 1024)){
    return String(bytes/1024.0/1024.0)+"MB";
  } //end else if
  else {
    return String(bytes/1024.0/1024.0/1024.0)+"GB";
  } //end else
} //end function formatBytes

//Content Type for File System
String getContentType(String filename)
{
  if(server.hasArg("download")) return "application/octet-stream";
  else if(filename.endsWith(".htm")) return "text/html";
  else if(filename.endsWith(".html")) return "text/html";
  else if(filename.endsWith(".css")) return "text/css";
  else if(filename.endsWith(".js")) return "application/javascript";
  else if(filename.endsWith(".png")) return "image/png";
  else if(filename.endsWith(".gif")) return "image/gif";
  else if(filename.endsWith(".jpg")) return "image/jpeg";
  else if(filename.endsWith(".ico")) return "image/x-icon";
  else if(filename.endsWith(".xml")) return "text/xml";
  else if(filename.endsWith(".pdf")) return "application/x-pdf";
  else if(filename.endsWith(".zip")) return "application/x-zip";
  else if(filename.endsWith(".gz")) return "application/x-gzip";

  if(mySerial)  
    mySerial.println("getContentType FUNC");

  return "text/plain";
} //end function getContentType

//File Read for File System
bool handleFileRead(String path)
{
  if(mySerial)  
    mySerial.println("handleFileRead: " + path);
  if(path.endsWith("/")) path += "index.htm";
  String contentType = getContentType(path);
  String pathWithGz = path + ".gz";
  if(SPIFFS.exists(pathWithGz) || SPIFFS.exists(path))
  {


    if(SPIFFS.exists(pathWithGz))
      path += ".gz";
    File file = SPIFFS.open(path, "r");

    if(mySerial)
        mySerial.println("TEST: " + path + " FILE OPEN: " + file.size());

    size_t sent = server.streamFile(file, contentType);

    if(mySerial)
        mySerial.println("TEST: " + path + " SIZE: " + file.size());

    file.close();

    if(mySerial)
        mySerial.println("TEST: " + path + " FILE CLOSE");

    return true;
  } //end if 
  return false;
} //end function handleFileRead

//File Upload for File System
void handleFileUpload(){
  if(server.uri() != "/edit") return;
  HTTPUpload& upload = server.upload();
  if(upload.status == UPLOAD_FILE_START)
  {
    String filename = upload.filename;
    if(!filename.startsWith("/")) 
        filename = "/"+filename;
    if(mySerial)
    {
        mySerial.print("handleFileUpload Name: "); 
        mySerial.println(filename);
    } //end if

    fsUploadFile = SPIFFS.open(filename, "w");
    filename = String();
  } //end if 
  else if(upload.status == UPLOAD_FILE_WRITE)
  {
    if(mySerial)
    {
        mySerial.print("handleFileUpload Data: "); 
        mySerial.println(upload.currentSize);
    } //end if

    if(fsUploadFile)
      fsUploadFile.write(upload.buf, upload.currentSize);
  } //end else if
  else if(upload.status == UPLOAD_FILE_END)
  {
    if(fsUploadFile)
      fsUploadFile.close();
    if(mySerial)
    {
        mySerial.print("handleFileUpload Size: "); 
        mySerial.println(upload.totalSize);
    } //end if
  } //end else if
} //end function handleFileUpload

//File Delete for File System
void handleFileDelete()
{
  if(server.args() == 0) return server.send(500, "text/plain", "BAD ARGS");
  String path = server.arg(0);
  if(mySerial)
    mySerial.println("handleFileDelete: " + path);
  if(path == "/")
    return server.send(500, "text/plain", "BAD PATH");
  if(!SPIFFS.exists(path))
    return server.send(404, "text/plain", "FileNotFound");
  SPIFFS.remove(path);
  server.send(200, "text/plain", "");
  path = String();
} //end function handleFileDelete

//File Create for File System
void handleFileCreate()
{
  if(server.args() == 0)
    return server.send(500, "text/plain", "BAD ARGS");
  String path = server.arg(0);
  if(mySerial)
      mySerial.println("handleFileCreate: " + path);
  if(path == "/")
    return server.send(500, "text/plain", "BAD PATH");
  if(SPIFFS.exists(path))
    return server.send(500, "text/plain", "FILE EXISTS");
  File file = SPIFFS.open(path, "w");
  if(file)
    file.close();
  else
    return server.send(500, "text/plain", "CREATE FAILED");
  server.send(200, "text/plain", "");
  path = String();
} //end function handleFileCreate

void handleFileList() 
{
  if(!server.hasArg("dir")) 
  {
      server.send(500, "text/plain", "BAD ARGS"); 
      return;
  } //end if

  String path = server.arg("dir");
  if(mySerial)
    mySerial.println("handleFileList: " + path);
  Dir dir = SPIFFS.openDir(path);
  path = String();

  String output = "[";
  while(dir.next())
  {
    File entry = dir.openFile("r");
    if (output != "[") output += ',';
    bool isDir = false;
    output += "{\"type\":\"";
    output += (isDir)?"dir":"file";
    output += "\",\"name\":\"";
    output += String(entry.name()).substring(1);
    output += "\"}";
    entry.close();
  } //end while

  output += "]";
  server.send(200, "text/json", output);
} //end function handleFileList

//HHIO PTT POWER LED ON
void switchPOWERon() {

    int powerStatusLength = 1;

    subsys = SUBSYS_1; 

    //Account for the end of message
    messageLength = powerStatusLength + 2;

    messageContent = 1;

    Serial.write(som);
    Serial.write(messageLength);
    Serial.write(subsys);
    Serial.write(messageContent);
    Serial.write(eom);
} //end switchPOWERon

//HHIO PTT POWER LED OFF
void switchPOWERoff() {

    int powerStatusLength = 1;

    subsys = SUBSYS_1; 

    //Account for the end of message
    messageLength = powerStatusLength + 2;

    messageContent = 0;

    Serial.write(som);
    Serial.write(messageLength);
    Serial.write(subsys);
    Serial.write(messageContent);
    Serial.write(eom);
} //end switchPOWERoff

//PUMP POWER ON
void pumpPOWERon() {

    int pumpPowerStatusLength = 1;

    subsys = SUBSYS_2; 

    //Account for the end of message
    messageLength = pumpPowerStatusLength + 2;

    messageContent = 1;

    Serial.write(som);
    Serial.write(messageLength);
    Serial.write(subsys);
    Serial.write(messageContent);
    Serial.write(eom);
} //end switchPOWERon

//PUMP POWER OFF
void pumpPOWERoff() {

    int pumpPowerStatusLength = 1;

    subsys = SUBSYS_2; 

    //Account for the end of message
    messageLength = pumpPowerStatusLength + 2;

    messageContent = 0;

    Serial.write(som);
    Serial.write(messageLength);
    Serial.write(subsys);
    Serial.write(messageContent);
    Serial.write(eom);
} //end switchPOWERoff

//LED POWER ON
void switchLEDon() {

    int ledStatusLength = 1;

    subsys = SUBSYS_3; 

    //Account for the end of message
    messageLength = ledStatusLength + 2;

    messageContent = 1;

    Serial.write(som);
    Serial.write(messageLength);
    Serial.write(subsys);
    Serial.write(messageContent);
    Serial.write(eom);
} //end switchLEDon

//LED POWER OFF
void switchLEDoff() {

    int ledStatusLength = 1;

    subsys = SUBSYS_3; 

    //Account for the end of message
    messageLength = ledStatusLength + 2;

    messageContent = 0;

    Serial.write(som);
    Serial.write(messageLength);
    Serial.write(subsys);
    Serial.write(messageContent);
    Serial.write(eom);
} //end switchLEDoff

//Function to send all updated status values from arduino to the webpage
void statusUpdate(uint8_t num) {

    //Valve C
    if(currValveStatusNumC == 0)
    {
        valveColorC = "red";
        valveTubePropsC = "175px solid #00ADEF";
        valveStatusC = "Closed";
    } //end if
    else if(currValveStatusNumC == 1)
    {
        valveColorC = "green";
        valveTubePropsC = "350px solid #00ADEF"; 
        valveStatusC = "Open";
    } //end else if

    //Valve D
    if(currValveStatusNumD == 0)
    {
        valveColorD = "red";
        valveTubePropsD = "175px solid #00ADEF";
        valveStatusD = "Closed";
    } //end if
    else if(currValveStatusNumD == 1)
    {
        valveColorD = "green";
        valveTubePropsD = "350px solid #00ADEF"; 
        valveStatusD = "Open";
    } //end else if

    //Valve E
    if(currValveStatusNumE == 0)
    {
        valveColorE = "red";
        valveTubePropsE = "175px solid #00ADEF";
        valveStatusE = "Closed";
    } //end if
    else if(currValveStatusNumE == 1)
    {
        valveColorE = "green";
        valveTubePropsE = "350px solid #00ADEF"; 
        valveStatusE = "Open";
    } //end else if

    String test = ""; 
    test += currNumRefresh;
    test += ",";
    test += currMicroC;
    test += ",";
    test += currMicroD;
    test += ",";
    test += currMicroE;
    test += ",";
    test += currPressureC;
    test += ","; 
    test += currPressureD;
    test += ",";
    test += currPressureE;
    test += ","; 
    test += valveColorC;
    test += ",";
    test += valveTubePropsC;
    test += ",";
    test += valveStatusC;
    test += ","; 
    test += valveColorD;
    test += ",";
    test += valveTubePropsD;
    test += ",";
    test += valveStatusD;
    test += ","; 
    test += valveColorE;
    test += ",";
    test += valveTubePropsE;
    test += ",";
    test += valveStatusE;
    test += ",";
    test += currFluid;

    test.toCharArray(statusbuf, 256);
    webSocket.sendTXT(num, statusbuf, strlen(statusbuf));
} //end function statusUpdate

// Current POWER status
bool POWERStatus;

// Current LED status
bool LEDStatus;

// Commands sent through Web Socket
const char LEDON[] = "ledon";
const char LEDOFF[] = "ledoff";

const char teensyPOWERON[] = "teensyPOWERon";
const char teensyPOWEROFF[] = "teensyPOWERoff";

const char pumpPOWERON[] = "pumpPOWERon";
const char pumpPOWEROFF[] = "pumpPOWERoff";

const char teensyLEDON[] = "teensyLEDon";
const char teensyLEDOFF[] = "teensyLEDoff";

const char statusIdentifier[] = "Update Status";


//Websocket Event Function
void webSocketEvent(uint8_t num, WStype_t type, uint8_t * payload, size_t length)
{
  //if(mySerial)
   //mySerial.printf("webSocketEvent(%d, %d, ...)\r\n", num, type);
  switch(type) {
    case WStype_DISCONNECTED:
      //if(mySerial)
          //mySerial.printf("[%u] Disconnected!\r\n", num);
      break;
    case WStype_CONNECTED:
      {
        IPAddress ip = webSocket.remoteIP(num);
        //if(mySerial)
          //mySerial.printf("[%u] Connected from %d.%d.%d.%d url: %s\r\n", num, ip[0], ip[1], ip[2], ip[3], payload);

      } //end case CONNECTED
      break;
    case WStype_TEXT:
      //if(mySerial)
        //mySerial.printf("[%u] get Text: %s\r\n", num, payload);

      if(strcmp(teensyPOWERON,  (const char *)payload) == 0) {
          switchPOWERon();
      } //end if

      else if(strcmp(teensyPOWEROFF,  (const char *)payload) == 0) {
          switchPOWERoff();
      } //end else if

      else if(strcmp(pumpPOWERON,  (const char *)payload) == 0) {
          pumpPOWERon();
      } //end else if

      else if(strcmp(pumpPOWEROFF,  (const char *)payload) == 0) {
          pumpPOWERoff();
      } //end else if

      else if(strcmp(teensyLEDON,  (const char *)payload) == 0) {
          switchLEDon();
      } //end else if

      else if(strcmp(teensyLEDOFF,  (const char *)payload) == 0) {
          switchLEDoff();
      } //end else if

      else if(strcmp(statusIdentifier, (const char *)payload) == 0) {
          statusUpdate(num);
      } //end else if
      else 
      {
        if(mySerial)
            mySerial.println("Unknown command");
      } //end else

      // send data to all connected clients
      webSocket.broadcastTXT(payload, length);
      break;
    case WStype_BIN:
      if(mySerial)
        mySerial.printf("[%u] get binary length: %u\r\n", num, length);
      hexdump(payload, length);

      // echo data back to browser
      webSocket.sendBIN(num, payload, length);
      break;
    default:
      if(mySerial)
        mySerial.printf("Invalid WStype [%d]\r\n", type);
      break;
  } //end switch
} //end function webSocketEvent

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

[kiralikbeyin]

Hi,

Very nice code.

I have some suggestions;

1-sometimes mDNS makes trouble , reboot your router,pc and esp ---> because "ws://myword.local" in your webpage kills your connection so try to use;

var connection = new WebSocket('ws://192.168.1.23:81/', ['arduino']);

2-Dont use font css - remove them from header
3-If you will use fonts like font awesome never gz them
4-try to make js files .js.gz

Bonus here :)

Remove 3. line from handleFileRead

bool handleFileRead(String path) {
  Serial.println("handleFileRead: " + path);
 // if (path.endsWith("/")) path += "index.htm";

add this to your setup

  server.on("/", HTTP_GET, []() {
    if (!handleFileRead2("/index.htm")) server.send(404, "text/plain", "FileNotFound");
  });

add handleFileRead2

bool handleFileRead2(String path) {
  Serial.println("handleFileRead: " + path);
  if (path.endsWith("/")) path += "index.htm";
  String contentType = getContentType(path);
  String pathWithGz = path + ".gz";
  if (SPIFFS.exists(pathWithGz) || SPIFFS.exists(path)) {
    if (SPIFFS.exists(pathWithGz))
      path += ".gz";
    File file = SPIFFS.open(path, "r");
     size_t sent = server.streamFile(file, contentType);
    file.close();
    return true;
  }
  return false;
}

UPDATE:

Merge all of your js files to only one js file and also css files to one css file

I hope this helps.

[Th3On3Fr33man]

@kiralikbeyin I don't know why so much of my code isn't being shown the way it should be in this new reply, I used the code tags :(

alright so I ended up switching over to that modified version of the library in the meantime since I made this post, and unfortunately I don't think I can apply your current solution directly since it relies on the original library.

Sorry but is there any chance that we could find a way to apply your solution to my new code?

Here is the relevant set of functions:

` class SPIFFSEditor: public AsyncWebHandler {
private:
String _username;
String _password;
bool _uploadAuthenticated;
public:
SPIFFSEditor(String username=String(), String password=String()):_username(username),_password(password),_uploadAuthenticated(false){}
bool canHandle(AsyncWebServerRequest *request){
if(request->method() == HTTP_GET && request->url() == "/edit" && (SPIFFS.exists("/edit.htm") || SPIFFS.exists("/edit.htm.gz")))
return true;
else if(request->method() == HTTP_GET && request->url() == "/list")
return true;
else if(request->method() == HTTP_GET && (request->url().endsWith("/") || SPIFFS.exists(request->url()) || (!request->hasParam("download") && SPIFFS.exists(request->url()+".gz"))))
return true;
else if(request->method() == HTTP_POST && request->url() == "/edit")
return true;
else if(request->method() == HTTP_DELETE && request->url() == "/edit")
return true;
else if(request->method() == HTTP_PUT && request->url() == "/edit")
return true;
return false;
}

//Function used to serve webpage to user (Modified Server Library Code)
void handleRequest(AsyncWebServerRequest *request){
  if(_username.length() && (request->method() != HTTP_GET || request->url() == "/edit" || request->url() == "/list") && !request->authenticate(_username.c_str(),_password.c_str()))
    return request->requestAuthentication();

  if(request->method() == HTTP_GET && request->url() == "/edit"){
    request->send(SPIFFS, "/edit.htm");
  } else if(request->method() == HTTP_GET && request->url() == "/list"){
    if(request->hasParam("dir")){
      String path = request->getParam("dir")->value();
      Dir dir = SPIFFS.openDir(path);
      path = String();
      String output = "[";
      while(dir.next()){
        File entry = dir.openFile("r");
        if (output != "[") output += ',';
        bool isDir = false;
        output += "{\"type\":\"";
        output += (isDir)?"dir":"file";
        output += "\",\"name\":\"";
        output += String(entry.name()).substring(1);
        output += "\"}";
        entry.close();
      }
      output += "]";
      request->send(200, "text/json", output);
      output = String();
    }
    else
      request->send(400);
  } else if(request->method() == HTTP_GET){
    String path = request->url();
    if(path.endsWith("/"))
      path += "index.htm";
    request->send(SPIFFS, path, String(), request->hasParam("download"));
  } else if(request->method() == HTTP_DELETE){
    if(request->hasParam("path", true)){
      ESP.wdtDisable(); SPIFFS.remove(request->getParam("path", true)->value()); ESP.wdtEnable(10);
      request->send(200, "", "DELETE: "+request->getParam("path", true)->value());
    } else
      request->send(404);
  } else if(request->method() == HTTP_POST){
    if(request->hasParam("data", true, true) && SPIFFS.exists(request->getParam("data", true, true)->value()))
      request->send(200, "", "UPLOADED: "+request->getParam("data", true, true)->value());
    else
      request->send(500);
  } else if(request->method() == HTTP_PUT){
    if(request->hasParam("path", true)){
      String filename = request->getParam("path", true)->value();
      if(SPIFFS.exists(filename)){
        request->send(200);
      } else {
        File f = SPIFFS.open(filename, "w");
        if(f){
          f.write(0x00);
          f.close();
          request->send(200, "", "CREATE: "+filename);
        } else {
          request->send(500);
        }
      }
    } else
      request->send(400);
  }
}

//Function used to handle uploading web files (Modified Server Library Code)
void handleUpload(AsyncWebServerRequest *request, String filename, size_t index, uint8_t *data, size_t len, bool final){
  if(!index){
    if(!_username.length() || request->authenticate(_username.c_str(),_password.c_str()))
      _uploadAuthenticated = true;
    request->_tempFile = SPIFFS.open(filename, "w");
  }
  if(_uploadAuthenticated && request->_tempFile && len){
    ESP.wdtDisable(); request->_tempFile.write(data,len); ESP.wdtEnable(10);
  }
  if(_uploadAuthenticated && final)
    if(request->_tempFile) request->_tempFile.close();
}

};`

I can see one of the lines you asked to remove is in here: // if (path.endsWith("/")) path += "index.htm";

And here is my new setup as well:

`void setup(){

IPAddress ip(192, 168, 1, 16);
IPAddress subnet(255, 255, 255, 0);
IPAddress gt(192, 168, 1, 1);

//Initialize Serial Connections
initSerial();

//Initialize File System
SPIFFS.begin();

//Initialize WiFi Connection (Modified Server Library Code)
//WiFi.mode(WIFI_STA);
WiFi.config(ip, gt, subnet);
WiFi.begin(ssid, password);
if (WiFi.waitForConnectResult() != WL_CONNECTED) {

Serial.printf("STA: Failed!\n");
WiFi.disconnect(false);

//Seems to work without this, not sure if necessary
//delay(1000);

WiFi.begin(ssid, password);

}
ArduinoOTA.begin();
//Serial.printf("format start\n"); SPIFFS.format(); Serial.printf("format end\n");

//NEW
ws.onEvent(onEvent);
server.addHandler(&ws);

server.serveStatic("/fs", SPIFFS, "/");

server.on("/heap", HTTP_GET, [](AsyncWebServerRequest *request){
request->send(200, "text/plain", String(ESP.getFreeHeap()));
});
server.addHandler(new SPIFFSEditor(http_username,http_password));

server.onNotFound([](AsyncWebServerRequest *request){

if(mySerial) 
{
    mySerial.printf("NOT_FOUND: ");
} //end if

if(request->method() == HTTP_GET)
{
    if(mySerial)
    {
        mySerial.printf("GET");
    } //end if
} //end if

else if(request->method() == HTTP_POST)
{
    if(mySerial)
    {
        mySerial.printf("POST");
    } //end if
} //end else if

else if(request->method() == HTTP_DELETE)
{
    if(mySerial)
    {
        mySerial.printf("DELETE");
    } //end if
} //end else if

else if(request->method() == HTTP_PUT)
{
    if(mySerial)
    {
        mySerial.printf("PUT");
    } //end if
} //end else if

else if(request->method() == HTTP_PATCH)
{
    if(mySerial)
    {
        mySerial.printf("PATCH");
    } //end if
} //end else if

else if(request->method() == HTTP_HEAD)
{
    if(mySerial)
    {
        mySerial.printf("HEAD");
    } //end if
} //end else if

else if(request->method() == HTTP_OPTIONS)
{
    if(mySerial)
    {
        mySerial.printf("OPTIONS");
    } //end if
} //end else if

else
{
    if(mySerial)
    {
        mySerial.printf("UNKNOWN");
    } //end if
} //end else

//Continued Server Initialization (Modified Server Library Code)
if(mySerial)
{
    mySerial.printf(" http://%s%s\n", request->host().c_str(), request->url().c_str());
} //end if

if(request->contentLength()){
    if(mySerial)
    {
        mySerial.printf("_CONTENT_TYPE: %s\n", request->contentType().c_str());
        mySerial.printf("_CONTENT_LENGTH: %u\n", request->contentLength());
    } //end if
} //end if

int headers = request->headers();
int i;
for(i=0;i<headers;i++){
  AsyncWebHeader* h = request->getHeader(i);
  if(mySerial)
  {
      mySerial.printf("_HEADER[%s]: %s\n", h->name().c_str(), h->value().c_str());
  } //end if
}

int params = request->params();
for(i=0;i<params;i++){
  AsyncWebParameter* p = request->getParam(i);
  if(p->isFile()){
    if(mySerial)
    {
        mySerial.printf("_FILE[%s]: %s, size: %u\n", p->name().c_str(), p->value().c_str(), p->size());
    } //end if
  } else if(p->isPost()){
    if(mySerial)
    {
        mySerial.printf("_POST[%s]: %s\n", p->name().c_str(), p->value().c_str());
    } //end if
  } else {
    if(mySerial)
    {
        mySerial.printf("_GET[%s]: %s\n", p->name().c_str(), p->value().c_str());
    } //end if
  }
}

request->send(404);

});
server.onFileUpload([](AsyncWebServerRequest *request, String filename, size_t index, uint8_t *data, size_t len, bool final){
if(!index)
{
if(mySerial)
{
mySerial.printf("UploadStart: %s\n", filename.c_str());
} //end if
} //end if

if(mySerial)
{
    mySerial.printf("%s", (const char*)data);
} //end if

if(final)
{
    if(mySerial)
    {
        mySerial.printf("UploadEnd: %s (%u)\n", filename.c_str(), index+len);
    } //end if
} //end if

});
server.onRequestBody([](AsyncWebServerRequest *request, uint8_t *data, size_t len, size_t index, size_t total){
if(!index)
{
if(mySerial)
{
mySerial.printf("BodyStart: %u\n", total);
} //end if
} //end if

if(mySerial)
{
    mySerial.printf("%s", (const char*)data);
} //end if

if(index + len == total)
{
    if(mySerial)
    {
        mySerial.printf("BodyEnd: %u\n", total);
    } //end if
} //end if

});
server.begin();

//Setup a timer for all WiFi status updates
statusTimerId = timer.setInterval(5000, sendInfoWifi);
}`

Sorry again and thank you so much for your assistance.

[kiralikbeyin]

There is nothing to do with esp code. I think all problem is reading a lot of files (size not important) More count more trouble.

Merge all of your js files to only one js file and also css files to one css file this will absolute solution.
(I merged 4 js files to 2 js files and also 5 css to 1 css file)

If you are using font awesome or gly of bootstrap they are raising the count of read files when startup. So don't use them now.(ERR_CONTENT_LENGTH_MISMATCH) +Some links in css code may search other css files or font files in dir, this is why you got this error.

Also try safari,chrome and firefox for console logs, they are giving different logs. Remember to clear browser cache before trusting your code.

+I am not using Async lib but it will work for you.

[Th3On3Fr33man]

@kiralikbeyin Alright I merged three of my css files into one and I took a gauge library I was using in js and merged it with my general js file.

However, there's another js file that I've had to include at the end of the file because it references elements in the html and creates elements that load at the start of the page. I'm not sure how to include this with my other js because it has to load later?

`var voltageGauge = new Gauge({
renderTo : 'voltageGauge',
width : 300,
height : 300,
glow : false,
units : 'V',
title : 'Voltage',
minValue : 0,
maxValue : 100,
majorTicks : ['0','10','20','30','40','50','60','70','80','90','100'],
minorTicks : 10,
strokeTicks : false,
highlights : [
{ from : 0, to : 50, color : 'rgba(0, 0, 255, .3)' },
{ from : 50, to : 100, color : 'rgba(255, 0, 0, .3)' }
],
colors : {
plate: '#222',
majorTicks: '#f5f5f5',
minorTicks: '#ddd',
title: '#fff',
units: '#ccc',
numbers: '#eee',
needle : {
start : 'rgba(240, 128, 128, 1)',
end : 'rgba(255, 160, 122, .9)',
circle: {
outerStart: '#333',
outerEnd: '#111',
innerStart: '#111',
innerEnd: '#222'
},
shadowUp: false,
shadowDown: false
},
circle: {
shadow: false,
outerStart: '#333',
outerEnd: '#111',
middleStart: '#222',
middleEnd: '#111',
innerStart: '#111',
innerEnd: '#333'
},
valueBox: {
rectStart: '#222',
rectEnd: '#333',
background: '#babab2',
shadow: 'rgba(0, 0, 0, 1)'
}
},
animation : {
delay : 25,
duration: 1000,
fn : 'bounce'
}
});

voltageGauge.draw();`

[kiralikbeyin]

Here Jquery before bootstrap,

at the end some custom code.

You can also directly paste js code before body close

        </div>
      </div>
    </div>
  </div>
  <script type="text/javascript" src="js/mycode.js"></script>
</body>

If you can't merge It is not a rule to use 1 js file, use it like;

  <script type="text/javascript" src="js/mycode.js"></script>
  <script type="text/javascript" src="js/mycode2.js"></script>

less file more performance.

Don't try to put css code to js file. Only merge css to css , js to js.

[Th3On3Fr33man]

Alright so I seemed to have mostly solved this issue by using the modified library, compressing my code, and removing jQuery entirely. I still get errors every once in a while, but these appear to be unavoidable and everything works most of the time.

@kiralikbeyin I compressed all my scripts into one and posted it at the end of the body as you suggested, which seems to work. Thanks for all your help.

[devyte]

@Th3On3Fr33man your description starts with ESP8266Webserver, but then later you discuss ESPAsyncWebserver.
Given your original code, and from personal experience, I think your original issue is in ESP8266Webserver.
Is the issue still relevant with

• latest git
• PR lwip2: lighter implementation #3362 (lwip2)
  ?

[devyte]

Closing due to lack of feedback and age.

[air18ru]

I have a slightly different problem, but its solution may suit you.
I use material design on my web server. This involves downloading large js and css files. The page took a long time to load - up to 7 seconds. I wondered why.
And found your issues. I noticed in your message the HTTP_DOWNLOAD_UNIT_SIZE parameter, which is set to 1460 bytes. I thought, why 1460? Perhaps this is to ensure that the data blocks fit well into the protocol data unit (PDU). But the low channel layer is able to split the data into units by itself. And there is no need to do it with a web server.
I set the value HTTP_DOWNLOAD_UNIT_SIZE to 16384 bytes. A miracle! The page began to load 4 times faster.
So, you may have a problem because some bytes are lost due to incorrect fragmentation, first by the web server, then by the channel layers.