parsing object mangles the input buffer and fails to parse, but parse returns true!

[sticilface]

I've written a class that parses a SPIFFS file into a buffer, parses it into an abject and returns it for the program to use. the buffer is a unique_ptr kept until the jsonpackage is out of scope... but for some reason parsing the buffer mangles it, the parse fails, but it returns true!

here it the code

int JSONpackage::parseSPIFS(const char * file, FS & fs) {

        //using namespace ESPMAN;
        File f = fs.open(file, "r");
        int totalBytes = f.size();
        if (!f) {
                return -1;
        }
        if (totalBytes > MELVANA_MAX_BUFFER_SIZE) {
                f.close();
                return -2;
        }

        _data = std::unique_ptr<char[]>(new char[totalBytes]);

        if (!_data) {
                f.close();
                return -3;
        }
        int position = 0;
        int bytesleft = totalBytes;
        while ((f.available() > -1) && (bytesleft > 0)) {
                // get available data size
                int sizeAvailable = f.available();

                if (sizeAvailable) {
                        int readBytes = sizeAvailable;

                        // read only the asked bytes
                        if (readBytes > bytesleft) {
                                readBytes = bytesleft;
                        }

                        // get new position in buffer
                        char * buf = &_data.get()[position];
                        // read data
                        int bytesread = f.readBytes(buf, readBytes);
                        if (readBytes && bytesread == 0) { break; } //  this fixes a corrupt file that has size but can't be read.
                        bytesleft -= bytesread;
                        position += bytesread;
                        Serial.printf("totalbytes = %u, sizeAvailable = %u, readBytes = %u, bytesleft = %u, position = %u\n", totalBytes, sizeAvailable, readBytes, bytesleft, position);
                        Serial.println("chunk = ");
                        Serial.write(buf, bytesread);
                        Serial.println();

                }
                // time for network streams
                delay(0);
        }
        f.close();

        if (bytesleft) {
          Serial.printf("parseERROR: bytesleft = %u\n", bytesleft);
          return -5;
        }

        Serial.println("BUFFER before parse:");
        Serial.write(_data.get(), totalBytes);
        Serial.println();

        if (_isArray) {
                _root = _jsonBuffer.parseArray(_data.get(),totalBytes);
        } else {
                _root = _jsonBuffer.parseObject(_data.get(),totalBytes);
        }
        if (!_root.success()) {
                return -4;
        }

        Serial.println("BUFFER after parse:");
        Serial.write(_data.get(), totalBytes);
        Serial.println();

        Serial.println("_root =");
        _root.prettyPrintTo(Serial);
        Serial.println();

        return 0;

}

here is the debug output

totalbytes = 256, sizeAvailable = 256, readBytes = 256, bytesleft = 0, position = 256
chunk = 
{
  "globals": {
    "MQTT": {
      "enabled": true,
      "ip": [
        192,
        168,
        1,
        10
      ],
      "port": 1883,
      "topics": [
        "home/downstairs/lights/+/set"
      ]
    },
    "pixels": 20
  }
}
BUFFER before parse:
{
  "globals": {
    "MQTT": {
      "enabled": true,
      "ip": [
        192,
        168,
        1,
        10
      ],
      "port": 1883,
      "topics": [
        "home/downstairs/lights/+/set"
      ]
    },
    "pixels": 20
  }
}
BUFFER after parse:
globals.obals": {
    "MQTT": {
      "enabled": true,
      "ip": [
        192,
        168,
        1,
        10
      ],
      "port": 1883,
      "topics": [
        "home/downstairs/lights/+/set"
      ]
    },
    "pixels": 20
  }
}
_root =
{}

From the output you can see that the file is read correctly into the buffer... you can see the buffer before and after the parse... where before it is fine but after globals.obals": { is problematic... I've not got a clue about why this is happening... can you spot anything from my code?

here are the full files...

JsonPackage.h.txt
JsonPackage.cpp.txt

[sticilface]

it looks like that is behaving exactly as it should... having looked at some examples that do work.
however, why is the parse successful even when _root = {} but the input buffer is valid?

[bblanchon]

The mangling part is expected, it's how ArduinoJson works. "It's not a bug, it's a feature" 😛

However, you found a bug in JsonVariant::success() which didn't propagate JsonObject::success().

Now, you might be asking why the parse fails... I bet that's because the JsonBufferran out of space.

[bblanchon]

The fix is available in v5.6.6.
Please open a new issue if there is more with this problem.
Thank you for finding that bug 👍

[sticilface]

cool thanks. yes i figured looking at successful parses that it is normal. i guess it stopped parsing and as you identified success was returning true. so yes a bug, glad to help.

TBH i've no idea why it was failing. it has started working, so may well be buffer related. i thought it might be the + sign, but it wasn't. The issue was that it was returning success!

Thanks for the hard work!