HTTPClient setConnectTimeout issues.

[CelliesProjects]

Board

ESP32 Dev Module

Device Description

Plain Devkit

Hardware Configuration

No

Version

v2.0.4

IDE Name

Arduino IDE

Operating System

Ubuntu something

Flash frequency

40Mhz

PSRAM enabled

no

Upload speed

460800

Description

@me-no-dev
The setConnectTimeout() in HTTPClient -since 2.0.3 or 2.0.2- still has the problem that setConnectTimeout() is not honored. I saw some commits and PRs #6676 and TD-er/ESPEasy@2c0ed04 that looked like they addressed the issue, but the issue is still there.

I checked out the HTTPClient source, but I don't have enough knowledge of the matter to solve it myself.

The following line of code in my current project does nothing usable since 2.0.3
_http->setConnectTimeout(url.startsWith("https") ? 2500 : 250);

And returns the following error:

[228558][I][WiFiClient.cpp:253] connect(): select returned due to timeout 250 ms for fd 48
[228558][W][HTTPClient.cpp:1469] returnError(): error(-1): connection refused
[228562][E][ESP32_VS1053_Stream.cpp:208] connecttohost(): [VS1053_Stream] error -1

When I check the time passed between the http request and error it is in the order of 60-65ms so something goes wrong there.

If I comment that line out, the connection is made in -again- 60-65ms.

Anybody recognise this, or better yet have a solution?

Sketch

/**
    BasicHTTPClient.ino

    Created on: 24.05.2015

*/

#include <Arduino.h>

#include <WiFi.h>
#include <WiFiMulti.h>

#include <HTTPClient.h>

#define USE_SERIAL Serial

WiFiMulti wifiMulti;

void setup() {

    USE_SERIAL.begin(115200);

    USE_SERIAL.println();
    USE_SERIAL.println();
    USE_SERIAL.println();

    for (uint8_t t = 4; t > 0; t--) {
        USE_SERIAL.printf("[SETUP] WAIT %d...\n", t);
        USE_SERIAL.flush();
        delay(1000);
    }

    wifiMulti.addAP("SSID", "PSK");

}

void loop() {
    // wait for WiFi connection
    if ((wifiMulti.run() == WL_CONNECTED)) {

        HTTPClient http;

        USE_SERIAL.print("[HTTP] begin...\n");


        http.setConnectTimeout(250);

        const unsigned long START_TIME = millis();
        http.begin("http://example.com/index.html"); //HTTP

        USE_SERIAL.print("[HTTP] GET...\n");
        // start connection and send HTTP header
        int httpCode = http.GET();

        USE_SERIAL.printf("[DEBUG]time elapsed since start of request: %i ms\n\n\n", millis() - START_TIME);

        // httpCode will be negative on error
        if (httpCode > 0) {
            // HTTP header has been send and Server response header has been handled
            USE_SERIAL.printf("[HTTP] GET... code: %d\n", httpCode);

            // file found at server
            if (httpCode == HTTP_CODE_OK) {
                String payload = http.getString();
                USE_SERIAL.println(payload);
            }
        } else {
            USE_SERIAL.printf("[HTTP] GET... failed, error: %s\n", http.errorToString(httpCode).c_str());
        }

        http.end();
    }

    delay(5000);
}

Debug Message

[HTTP] begin...
[HTTP] GET...
[120156][I][WiFiClient.cpp:253] connect(): select returned due to timeout 250 ms for fd 48
[120157][W][HTTPClient.cpp:1469] returnError(): error(-1): connection refused
[DEBUG]time elapsed since start of request: 6 ms


[HTTP] GET... failed, error: connection refused

Other Steps to Reproduce

No response

I have checked existing issues, online documentation and the Troubleshooting Guide

• I confirm I have checked existing issues, online documentation and Troubleshooting guide.

[P-R-O-C-H-Y]

Hi @CelliesProjects, can you try it with changes from PR #6676 ?

[CelliesProjects]

@P-R-O-C-H-Y I can but it will take a couple of days before I have some time.
Thanks for the suggestion, I will report back.

[CelliesProjects]

@P-R-O-C-H-Y How do I checkout master in the arduino IDE environment?

[P-R-O-C-H-Y]

@CelliesProjects Take a look here in our docs. There is a part with manual installation for Linux/MacOS and Windows. If you need any help, let me know :)

[CelliesProjects]

@P-R-O-C-H-Y I could use some help. The PR is in your GH account. How do I pull that locally and use it? My git knowledge is very limited...

[P-R-O-C-H-Y]

@CelliesProjects The PR is in this Arduino-esp32 repo, you don't need to download my fork.

1. You need to have the manual installation as I mentioned in comment above. With that you are now on latest master branch.
2. Now you need to checkout the PR to your local repo.
   • you can use command git pull origin pull/6676/head , that should apply all changes from PR to the master
   • or you can download Github Desktop app and do it from there. In the app you add your already installed repo. Then open the PR in web browser, on top right there is < > code button, if you click that you can select Checkout with Github Desktop.

[schreibfaul1]

Here is a simple example to show that there is a problem with the timeout

#include <Arduino.h>
#include "WiFi.h"

char SSID[] = "XXX";
char PW[] =   "XXX";

char host[] = "laut.fm";

WiFiClient client;

void setup() {
  Serial.begin(115200);
  WiFi.begin(SSID, PW);
  while (WiFi.status() != WL_CONNECTED) delay(1500);

  // test case 1  okay
  uint32_t timestamp = millis();
  bool connected = client.connect(host, 80);
  if(connected) Serial.printf("connected in %d ms \n", millis() - timestamp);
  else Serial.println("Connection Failed");
  client.stop();

  // test case 2  not working
  timestamp = millis();
  connected = client.connect(host, 80, 200);
  if(connected) Serial.printf("connected in %d ms", millis() - timestamp);
  else Serial.println("Connection Failed");
  client.stop();
}

void loop() {
  // put your main code here, to run repeatedly:
}

test case 1 (without timeout) works perfect, output connected in 41 ms
test case 2 (timeout 200ms) not working, output Connection Failed

this worked until version 2.0.2

[CelliesProjects]

I just spend an hour trying to get the PR mentioned above on my laptop. I ended up with a lot of error messages so I probably f*cked that up.
I will retry again but the coming weeks I am very busy IRL.

Multiple libraries were found for "WiFi.h"
 Used: /home/cellie/Arduino/hardware/espressif/esp32/libraries/WiFi
 Not used: /home/cellie/arduino-1.8.19/libraries/WiFi
/home/cellie/Arduino/hardware/espressif/esp32/tools/xtensa-esp32-elf/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld: libraries/ESPAsyncWebServer/AsyncWebSocket.cpp.o:(.literal._ZN22AsyncWebSocketResponseC2ERK6StringP14AsyncWebSocket+0x10): undefined reference to `SHA1Init'
/home/cellie/Arduino/hardware/espressif/esp32/tools/xtensa-esp32-elf/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld: libraries/ESPAsyncWebServer/AsyncWebSocket.cpp.o:(.literal._ZN22AsyncWebSocketResponseC2ERK6StringP14AsyncWebSocket+0x14): undefined reference to `SHA1Update'
/home/cellie/Arduino/hardware/espressif/esp32/tools/xtensa-esp32-elf/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld: libraries/ESPAsyncWebServer/AsyncWebSocket.cpp.o:(.literal._ZN22AsyncWebSocketResponseC2ERK6StringP14AsyncWebSocket+0x18): undefined reference to `SHA1Final'
/home/cellie/Arduino/hardware/espressif/esp32/tools/xtensa-esp32-elf/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld: libraries/ESPAsyncWebServer/AsyncWebSocket.cpp.o: in function `AsyncWebSocketResponse::AsyncWebSocketResponse(String const&, AsyncWebSocket*)':
/home/cellie/Arduino/libraries/ESPAsyncWebServer/src/AsyncWebSocket.cpp:1272: undefined reference to `SHA1Init'
/home/cellie/Arduino/hardware/espressif/esp32/tools/xtensa-esp32-elf/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld: /home/cellie/Arduino/libraries/ESPAsyncWebServer/src/AsyncWebSocket.cpp:1273: undefined reference to `SHA1Update'
/home/cellie/Arduino/hardware/espressif/esp32/tools/xtensa-esp32-elf/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld: /home/cellie/Arduino/libraries/ESPAsyncWebServer/src/AsyncWebSocket.cpp:1274: undefined reference to `SHA1Final'
/home/cellie/Arduino/hardware/espressif/esp32/tools/xtensa-esp32-elf/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld: libraries/ESPAsyncWebServer/WebAuthentication.cpp.o:(.literal._ZL6getMD5PhtPc+0x4): undefined reference to `mbedtls_md5_starts'
/home/cellie/Arduino/hardware/espressif/esp32/tools/xtensa-esp32-elf/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld: libraries/ESPAsyncWebServer/WebAuthentication.cpp.o: in function `getMD5(unsigned char*, unsigned short, char*)':
/home/cellie/Arduino/libraries/ESPAsyncWebServer/src/WebAuthentication.cpp:73: undefined reference to `mbedtls_md5_starts'
collect2: error: ld returned 1 exit status
exit status 1

[mrengineer7777]

@CelliesProjects Consider using WiFiClient instead of HTTPClient. #7686 (arduino-esp32 v2.0.7) makes sure WiFiClient initial connection timeout works correctly. Other connection timeouts are still being worked on in #6676.

[jcomas]

Using WiFiClient with LilyGo T-Watch 2020 V3:

Works:
.- connected = client.connect(host, 80);
.- connected = client.connect(host, 80, 1000);

Connection time is below 100ms. But using timeout 100 or 500 doesn't work:
.- connected = client.connect(host, 80, 100);
.- connected = client.connect(host, 80, 500);

[mrengineer7777]

Are you using arduino-esp32 v2.0.7? Older versions of WiFiClient rounded the connection time to the nearest whole second.

[jcomas]

Are you using arduino-esp32 v2.0.7? Older versions of WiFiClient rounded the connection time to the nearest whole second.

2.0.6

[mrengineer7777]

Switch to v2.0.7, or manually apply the fix in #7686.

[VojtechBartoska]

@CelliesProjects any updates?

[CelliesProjects]

I just tested in 2.0.7 and it works like expected!
Thanks to all!

@VojtechBartoska @mrengineer7777 I don't have the thread link atm but I read up on this issue and there was discussion going on about making breaking changes wrt this issue. That is no longer the case?

[mrengineer7777]

@CelliesProjects You may be thinking of #6676

[CelliesProjects]

@mrengineer7777
@VojtechBartoska

Still having an issue with the timeout.

[   397][I][eStreamPlayer32_VS1053.ino:374] setup(): WiFi connected - IP 192.168.0.101
[   398][I][eStreamPlayer32_VS1053.ino:380] setup(): Waiting for NTP sync...
[  2420][I][eStreamPlayer32_VS1053.ino:385] setup(): Synced
[  2423][I][eStreamPlayer32_VS1053.ino:537] setup(): Webserver started
[  2423][I][eStreamPlayer32_VS1053.ino:48] playerTask(): Starting VS1053 codec...
[  6122][I][eStreamPlayer32_VS1053.ino:60] playerTask(): Ready to rock!
[  8859][I][ESP32_VS1053_Stream.cpp:160] connecttohost(): Time elapsed during connect: 55
[ 14736][I][ESP32_VS1053_Stream.cpp:160] connecttohost(): Time elapsed during connect: 48
[ 23712][I][ESP32_VS1053_Stream.cpp:160] connecttohost(): Time elapsed during connect: 36
[ 33262][I][ESP32_VS1053_Stream.cpp:160] connecttohost(): Time elapsed during connect: 93
[ 48245][E][WiFiClient.cpp:517] flush(): fail on fd 48, errno: 11, "No more processes"
[ 61374][E][WiFiGeneric.cpp:1476] hostByName(): DNS Failed for amoris.sytes.net.
[ 61374][W][HTTPClient.cpp:1469] returnError(): error(-1): connection refused
[ 61377][I][ESP32_VS1053_Stream.cpp:160] connecttohost(): Time elapsed during connect: 13238
[ 61385][E][ESP32_VS1053_Stream.cpp:233] connecttohost(): error -1 connection refused

First you see 4 resolving hosts and then a non resolving host.
The "No more processes" message is probably unrelated. Happens more often with no ill effects.

And then in WiFiGeneric.cpp

int WiFiGenericClass::hostByName(const char* aHostname, IPAddress& aResult)
{
    if (!aResult.fromString(aHostname))
    {
        ip_addr_t addr;
        aResult = static_cast<uint32_t>(0);
        waitStatusBits(WIFI_DNS_IDLE_BIT, 16000);
        clearStatusBits(WIFI_DNS_IDLE_BIT | WIFI_DNS_DONE_BIT);
        err_t err = dns_gethostbyname(aHostname, &addr, &wifi_dns_found_callback, &aResult);
        if(err == ERR_OK && addr.u_addr.ip4.addr) {
            aResult = addr.u_addr.ip4.addr;
        } else if(err == ERR_INPROGRESS) {
            waitStatusBits(WIFI_DNS_DONE_BIT, 15000);  //real internal timeout in lwip library is 14[s]
            clearStatusBits(WIFI_DNS_DONE_BIT);
        }
        setStatusBits(WIFI_DNS_IDLE_BIT);
        if((uint32_t)aResult == 0){
            log_e("DNS Failed for %s", aHostname);
        }
    }
    return (uint32_t)aResult != 0;
}

The timeout is still set to 15000 when the DNS resolution fails.
At least that is what I suspect? Or am I reading that code wrong?

I would expect this call to fail if it goes over the timeout set with setConnectTimeout().

[mrengineer7777]

@CelliesProjects PR 6676 hasn't been merged yet. I would look into that "no more processes" error. That sounds serious.

[VojtechBartoska]

Closing this as solved.