How to log more info on 422 unprocessable entity

[nickgieschen]

I get this a lot in development and can usually pretty quickly figure out what is wrong with my json payload. However, I'd love a way to see what exactly the validation error is when I get this error. Is there a way I can log what the specific schema validation error is on the server side?

[Garito]

Just the other day I had a pretty absurd issue about exactly that

My gui was doing a fetch request and thanks to the apple keyword crap, I set the content-type to applications/json (notice the applications instead of application)

The 422 error indicated that value wasn't a dict

I spend too much time till I noticed the additional s

This would be avoided with better 422 error or a way to debug it

[panla]

@nickgieschen
hello

catch exception

def register_exception(app: FastAPI):
    @app.exception_handler(RequestValidationError)
    async def validation_exception_handler(request: Request, exc: RequestValidationError):

        exc_str = f'{exc}'.replace('\n', ' ').replace('   ', ' ')
        # or logger.error(f'{exc}')
        logger.error(request, exc_str)
        content = {'status_code': 10422, 'message': exc_str, 'data': None}
        return JSONResponse(content=content, status_code=status.HTTP_422_UNPROCESSABLE_ENTITY)

after you can log error and get response

and you can see issue 394 @ruslankrivoshein

[nickgieschen]

@nickgieschen
hello

catch exception

def register_exception(app: FastAPI):
    @app.exception_handler(RequestValidationError)
    async def validation_exception_handler(request: Request, exc: RequestValidationError):

        exc_str = f'{exc}'.replace('\n', ' ').replace('   ', ' ')
        # or logger.error(f'{exc}')
        logger.error(request, exc_str)
        content = {'status_code': 10422, 'message': exc_str, 'data': None}
        return JSONResponse(content=content, status_code=status.HTTP_422_UNPROCESSABLE_ENTITY)

after you can log error and get response

and you can see issue 394 @ruslankrivoshein

Thanks. Can you tell me what logger that is that you can pass request as the first argument?

[panla]

def log_message(request: Request, e):

    logger.error('start error'.center(60, '*'))
    logger.error(f'{request.method} {request.url}')
    logger.error(f'error is {e}')
    logger.error('end error'.center(60, '*'))

log_message(request, exc_str)

or

logger.error(exc_str)

[Torxed]

The above examples are excerpts of handling-errors documentation.
For the sake of clarity to anyone ending up here after having the same issues as OP, here's the full code @panla scissored in:

import logging
from fastapi import FastAPI, Request, status
from fastapi.exceptions import RequestValidationError
from fastapi.responses import JSONResponse

app = FastAPI()

@app.exception_handler(RequestValidationError)
async def validation_exception_handler(request: Request, exc: RequestValidationError):
	exc_str = f'{exc}'.replace('\n', ' ').replace('   ', ' ')
	logging.error(f"{request}: {exc_str}")
	content = {'status_code': 10422, 'message': exc_str, 'data': None}
	return JSONResponse(content=content, status_code=status.HTTP_422_UNPROCESSABLE_ENTITY)

This will produce something along the lines of:

ERROR:root:<starlette.requests.Request object at 0x7fba45db99d0>: 3 validation errors for Request body -> X none is not an allowed value (type=type_error.none.not_allowed) body -> Y value is not a valid list (type=type_error.list) body -> Z value is not a valid list (type=type_error.list)

This error is not only bound to the header, but also if the data submitted does not match the BaseModel supplied to the API endpoint.

[zwessels]

content = {'status_code': 10422, 'message': exc_str, 'data': None

presume it should be 422?

[dancaugherty]

What is the significance of 10422 ?

[VBoB13]

The above examples are excerpts of handling-errors documentation. For the sake of clarity to anyone ending up here after having the same issues as OP, here's the full code @panla scissored in:

import logging
from fastapi import FastAPI, Request, status
from fastapi.exceptions import RequestValidationError
from fastapi.responses import JSONResponse

app = FastAPI()

@app.exception_handler(RequestValidationError)
async def validation_exception_handler(request: Request, exc: RequestValidationError):
	exc_str = f'{exc}'.replace('\n', ' ').replace('   ', ' ')
	logging.error(f"{request}: {exc_str}")
	content = {'status_code': 10422, 'message': exc_str, 'data': None}
	return JSONResponse(content=content, status_code=status.HTTP_422_UNPROCESSABLE_ENTITY)

This will produce something along the lines of:

ERROR:root:<starlette.requests.Request object at 0x7fba45db99d0>: 3 validation errors for Request body -> X none is not an allowed value (type=type_error.none.not_allowed) body -> Y value is not a valid list (type=type_error.list) body -> Z value is not a valid list (type=type_error.list)

This error is not only bound to the header, but also if the data submitted does not match the BaseModel supplied to the API endpoint.

This should be the standard implementation IMHO... a beginner like me had to do quite a lot of digging simply because the error wasn't expressive enough :P

[JustinGuese]

wouldn't this catch all errors instead of only 422? and what about the 10422 like @dancaugherty mentioned?

[Torxed]

This should be the standard implementation IMHO..

At least if --debug is given or something.
It takes a toll if you have a lot of errors being catched this way, but yea, it's a rabbit hole before you end up here and just copy paste it in. Which is problematic in it's own right.

[JustinGuese]

yeah i agree, especially because there's no real way to log it properly if you'll get a 422

[JustinGuese]

or it would be cool to define when creating the pydantic model...
like

class Person(BaseModel):
   name: str
   
   def on_422_error(error):
       logging.log etc

[panla]

@nickgieschen @Torxed @zwessels @dancaugherty @VBoB13 @JustinGuese

please see

codes

extensions/exceptions.py

from typing import Optional, Any, Dict

from fastapi import status
from starlette.exceptions import HTTPException

from conf.const import StatusCode
from .schema import SchemaMixin


class BaseHTTPException(HTTPException):
    MESSAGE = None
    STATUS_CODE = status.HTTP_400_BAD_REQUEST
    CODE = 40000

    def __init__(
            self,
            message: Any = None,
            code: int = None,
            headers: Optional[Dict[str, Any]] = None
    ) -> None:
        self.message = message or self.MESSAGE
        self.status_code = self.STATUS_CODE
        self.code = code or self.CODE
        self.detail = self.message
        self.headers = headers

    def __repr__(self) -> str:
        class_name = self.__class__.__name__
        return f"{class_name}(status_code={self.status_code!r}, code={self.code}, msg={self.message!r})"

    def response(self):
        return SchemaMixin(code=self.code, message=self.message, data=None).dict()


class BadRequest(BaseHTTPException):
    STATUS_CODE = status.HTTP_400_BAD_REQUEST
    CODE = StatusCode.bad_request


class Unauthorized(BaseHTTPException):
    STATUS_CODE = status.HTTP_401_UNAUTHORIZED
    CODE = StatusCode.unauthorized


class Forbidden(BaseHTTPException):
    STATUS_CODE = status.HTTP_403_FORBIDDEN
    CODE = StatusCode.forbidden


class NotFound(BaseHTTPException):
    STATUS_CODE = status.HTTP_404_NOT_FOUND
    CODE = StatusCode.not_found


class MethodNotAllowed(BaseHTTPException):
    STATUS_CODE = status.HTTP_405_METHOD_NOT_ALLOWED
    CODE = StatusCode.method_not_allowed


class Locked(BaseHTTPException):
    STATUS_CODE = status.HTTP_423_LOCKED
    CODE = StatusCode.locked

extensions/schemas.py

from typing import Optional, Any

from fastapi import Query
from pydantic import BaseModel
from pydantic.typing import NoneType

from conf.const import StatusCode, PaginateConst


class BadRequestSchema(BaseModel):
    code: int = StatusCode.bad_request
    message: str = ''
    data: NoneType = "null"


class UnauthorizedSchema(BaseModel):
    code: int = StatusCode.unauthorized
    message: str = ''
    data: NoneType = "null"


class ForbiddenSchema(BaseModel):
    code: int = StatusCode.forbidden
    message: str = ''
    data: NoneType = "null"


class NotFoundSchema(BaseModel):
    code: int = StatusCode.not_found
    message: str = ''
    data: NoneType = "null"


class ValidatorErrorSchema(BaseModel):
    code: int = StatusCode.validator_error
    message: str = ''
    data: NoneType = "null"


ErrorSchema = {
    400: {
        'model': BadRequestSchema,
        'description': 'bad_request'
    },
    401: {
        'model': UnauthorizedSchema,
        'description': 'unauthorized'
    },
    403: {
        'model': ForbiddenSchema,
        'description': 'forbidden'
    },
    404: {
        'model': NotFoundSchema,
        'description': 'not_found'
    },
    422: {
        'model': ValidatorErrorSchema,
        'description': 'request parameters validator'
    }
}


class SchemaMixin(BaseModel):
    code: int = 10000
    message: str = ''
    data: Optional[Any]


class NormalSchema(SchemaMixin):
    """"""

    data: Optional[str] = 'success'

conf/const.py

class StatusCode(object):
    success = 10000

    bad_request = 40000
    unauthorized = 40100
    forbidden = 40300
    not_found = 40400
    method_not_allowed = 40500
    not_acceptable = 40600
    request_timeout = 40800
    length_required = 41100
    entity_too_large = 41300
    request_uri_too_long = 41400
    validator_error = 42200
    locked = 42300
    header_fields_too_large = 43100

    server_error = 45000
    unknown_error = 45001

apps/libs/exceptions.py

import traceback
from typing import Union, Any

from fastapi import FastAPI, Request, status
from fastapi.exceptions import HTTPException, RequestValidationError
from fastapi.responses import JSONResponse
from starlette.datastructures import URL
from tortoise.validators import ValidationError

from conf.const import StatusCode
from extensions.log import logger
from extensions.exceptions import BaseHTTPException


def log_message(method: str, url: Union[str, URL], message: Any):
    """log message when catch exception"""

    logger.error('start error, this is'.center(60, '*'))
    logger.error(f'{method} {url}')
    logger.error(message)
    logger.error('end error'.center(60, '*'))


def register_exception(app: FastAPI):
    @app.exception_handler(BaseHTTPException)
    async def catch_c_http_exception(request: Request, exc: BaseHTTPException):
        """catch custom exception"""

        log_message(request.method, request.url, exc.message)
        content = exc.response()
        return JSONResponse(content=content, status_code=exc.status_code, headers=exc.headers)

    @app.exception_handler(HTTPException)
    async def http_exception_handler(request: Request, exc: HTTPException):
        """catch FastAPI HTTPException"""

        log_message(request.method, request.url, exc.detail)
        content = {'code': StatusCode.bad_request, 'message': exc.detail, 'data': None}
        return JSONResponse(content=content, status_code=exc.status_code, headers=exc.headers)

    @app.exception_handler(AssertionError)
    async def assert_exception_handle(request: Request, exc: AssertionError):
        """catch Python AssertError"""

        exc_str = ' '.join(exc.args)
        log_message(request.method, request.url, exc_str)
        content = {'code': StatusCode.validator_error, 'message': exc_str, 'data': None}
        return JSONResponse(content=content, status_code=status.HTTP_422_UNPROCESSABLE_ENTITY)

    @app.exception_handler(ValidationError)
    async def db_validation_exception_handle(request: Request, exc: ValidationError):
        """catch tortoise-orm ValidatorError"""

        exc_str = '|'.join(exc.args)
        log_message(request.method, request.url, exc.args)
        content = {'code': StatusCode.validator_error, 'message': exc_str, 'data': None}
        return JSONResponse(content=content, status_code=status.HTTP_422_UNPROCESSABLE_ENTITY)

    @app.exception_handler(RequestValidationError)
    async def validation_exception_handler(request: Request, exc: RequestValidationError):
        """catch FastAPI RequestValidationError"""

        exc_str = f'{exc}'.replace('\n', ' ').replace('   ', ' ')
        log_message(request.method, request.url, exc)
        # content = exc.errors()
        content = {'code': StatusCode.validator_error, 'message': exc_str, 'data': None}
        return JSONResponse(content=content, status_code=status.HTTP_422_UNPROCESSABLE_ENTITY)

    @app.exception_handler(Exception)
    async def exception_handle(request: Request, exc: Exception):
        """catch other exception"""

        log_message(request.method, request.url, traceback.format_exc())
        content = {'code': StatusCode.server_error, 'message': str(exc), 'data': None}
        return JSONResponse(content=content, status_code=status.HTTP_500_INTERNAL_SERVER_ERROR)

desc

success

http status code 200 or 201 or 204

response

{
    "code": 10000,
    "message": "success",
    "data": "how are you?"
}

422 error

http status code 422

response

{
    "code": 42200,
    "message": "validator error",
    "data": null
}

other response

use exception

from extensions.exceptions import NotFound, BadRequest

raise NotFound(message=f'there is no this user {cellphone}')
raise BadRequest(message='Request too fast!')

raise CustomException(message="", code=1000000000000, headers=None)

last

Don't pay too much attention to some definitions.

This is just a demo

Defined according to your actual situation

[vlori2k]

I see alot of answers here, which one is best for simply returning the field error to the user?

Such as:

"field" : pause is missing: ?

Here is my code:

@app.exception_handler(RequestValidationError)
async def validation_exception_handler(request: Request, exc: RequestValidationError):
	exc_str = f'{exc}'.replace('\n', ' ').replace('   ', ' ')
	#logging.error(f"{request}: {exc_str}")
	print(f"{request}: {exc_str}")
	content = {'status_code': 10422, 'message': exc_str, 'data': None}
	return JSONResponse(content=content, status_code=status.HTTP_422_UNPROCESSABLE_ENTITY)




@app.put("/edit_exercise_on_my_plan") #look at this as line 1 in code
async def edit_exercise_on_my_plan(workout_plan_task_delete: schemas.a_specific_workout_plan_task_edit, Authorize: AuthJWT = Depends()):

    Authorize.jwt_required()
try:
  a_user_controller = UserRegController(workout_plan_task_delete.email_address)
  
  if await a_user_controller.get_user_reg_info() == False:
      # no user registered.. show this
      return JSONResponse(status_code=share_this_class_around_project.error_codes_from_db_dict[28]["HTTP_code"], content={
          share_this_class_around_project.error_codes_from_db_dict[28]["message_code"]:
              share_this_class_around_project.error_codes_from_db_dict[28]["message"]})
  
  
  else:
      a_workout_plan_controller = WorkOutPlanController(a_user_controller.user_reg_ID)
  
      if await a_workout_plan_controller.check_if_user_subscribed_to_specific_plan(
              workout_plan_task_delete.workout_plan_ID) == False:
  
          return JSONResponse(status_code=share_this_class_around_project.error_codes_from_db_dict[91]["HTTP_code"],
                              content={share_this_class_around_project.error_codes_from_db_dict[91]["message_code"]:
                                           share_this_class_around_project.error_codes_from_db_dict[91]["message"]})
  
      else:
          print(" bla bla continue ..")

except Exception as e:
print("edit exercise on my plan POST, sup here?", e)

except RequestValidationError as lol:
print("can i print something here?", lol)

The problem is, that as soon as the scheme is not correct, i can not get further than line one, which means i can not use the RequestValidationError ?

[tiangolo]

Thanks for the help here everyone! 👏 🙇

If that solves the original problem, then you can close this issue @nickgieschen ✔️

Sorry for the long delay! 🙈 I wanted to personally address each issue/PR and they piled up through time, but now I'm checking each one in order.

[scratchmex]

@tiangolo can we add a logger in the exception handlers exception_handlers.py so that it logs the validation errors to stdout? This is useful if you are testing the API with another client and you can't see the response directly. Something like:

async def request_validation_exception_handler(
    request: Request, exc: RequestValidationError
) -> JSONResponse:
    logger.debug(f"[422] {exc.errors()}") <<<<<<<<
    return JSONResponse(
        status_code=HTTP_422_UNPROCESSABLE_ENTITY,
        content={"detail": jsonable_encoder(exc.errors())},
    )

so that I see on my terminal this

[422] [{'type': 'missing', 'loc': ('body', 'refresh_token'), 'msg': 'Field required', 'input': None, 'url': 'https://errors.pydantic.dev/2.5/v/missing'}]
INFO:     127.0.0.1:54360 - "POST /oauth2/token HTTP/1.1" 422 Unprocessable Entity

will be happy to open a PR on this

[jderrien]

I agree it should be logged by default.

FastAPI raises an exception, returns an error to the client, so it should also log the exception or the error on the server side.
IMHO, a debug log is not enough.

I'll go with:

logger.error("%s: %s", type(exc).__name__, exc)

It would give something like:

2024-01-19 11:39:11,279 fastapi ERROR: RequestValidationError: [{'type': 'int_parsing', 'loc': ('path', 'item_id'), 'msg': 'Input should be a valid integer, unable to parse string as an integer', 'input': 'foo', 'url': 'https://errors.pydantic.dev/2.5/v/int_parsing'}]
2024-01-19 11:39:11,279 uvicorn.access INFO:     192.168.65.1:23293 "GET /items/foo HTTP/1.1" 422 Unprocessable Entity

[thinkjrs]

For future travelers here's the section in the docs that covers this topic: https://fastapi.tiangolo.com/tutorial/handling-errors/#use-the-requestvalidationerror-body