How to hide certain fields in Pydantic models?

[enilsen16]

Example:

I have created_at and updated_at fields that I want to auto populate on the model using default_factory, but I don't want the user to be able to set them via an endpoint. I thought this could be achieved by the read only Field arg but then I can never set it.

What's the best way to go about this?

[includeamin]

from pydantic import BaseModel,Field
from datetime import datetime



class User(BaseModel):
    name:str
    create_at: datetime = datetime.now()

user_in_system = User(name='includeamin')
print(user_in_system.dict(exclude={'create_at'}))
# output: {'name': 'includeamin'}

[enilsen16]

Sorry I should have clarified, how do you hide them in the openapi spec? :)

[phy25]

I mean you could create two models, one internal and one external, but you have to be careful between converting between two models (performance v.s. validation).

[enilsen16]

Yeah it doesn't seem like there is a good way to convert between two models without validation?

[enilsen16]

Sorry I should have clarified, how do you hide them in the openapi spec? :)

I did notice that Pydantic has a schema_extra function that you can provide but fast didn't seem to be picking it up when generating the openapi docs.

[tiangolo]

Check the docs here: https://fastapi.tiangolo.com/tutorial/response-model/#add-an-output-model

Having a model declared as having an attribute and then making it not have the attribute in some other way would make your editor, linter (and you) believe that that model has that attribute while it doesn't. So it's better/easier to declare the 2 models that inherit from the same base.

[takeda]

I'm having similar issue. The example works well for simple cases, but this approach becomes problematic when nested models are used and you want to control fields of one or two classes at the bottom. I would have to also rewrite all the classes that are encompassing them, even though nothing is different with them :(

[a-recknagel]

Hide where? In the API docs, when calling .json() on the top level model, or something else?

[takeda]

Hide from API and hide from dict() call.

I basically have a multiclass structure that is used in 3 places.

Just to give a background. The application sets up a private link in aws, and returns its endpoint as well as health check to the user.

• API (returns health check at the time of invocation, that isn't stored in dynamodb, I don't want to return target groups that I store in dynamodb)
• the record stored in dynamodb (the health check has no meaning at that state so I don't want to store it, but I want to store target groups)
• used to pass data to a job (this one is least problematic, I currently don't do any serialization so can just ignore extra fields)

I was basically had 3 separate class hierarchies and introduced functions to transform from one to the other. This works, but I realized this makes the code hard to maintain when new fields need to be added, and there's a huge room for mistakes, so I am trying to consolidate it.

I think what would work from me, would be maybe something like:

class A(BaseModel):
  field_a: str
  field_b: str
  field_c: int
  field_d: str
  
  class Config:
    exclude_api = {"field_b", "field_c"}
    exclude_groups = dict(
      dynamodb={"field_b"},
      batch={"field_c"}
    )
  
class Response(BaseModel):
  field_a: A
  field_b: str

  class Config:
    exclude_from = dict(
      dynamodb={"field_b"},
    )

then I could just use obj.dict(exclude_grup="dynamodb").

[github-actions[bot]]

Assuming the original issue was solved, it will be automatically closed now. But feel free to add more comments or create new issues.

[serain]

@enilsen16 did you ever find a clean way to hide a particular field from the schema?

In my case I have some required headers that are set downstream by my ingress. FastAPI expects to see those headers but it's confusing for my users to see them in the docs.

[aberrier]

Pydantic allows you to define extra arguments on Field.
You can use this at your advantage like so

from pydantic import BaseModel, Field


class CustomBaseModel(BaseModel):
    def dict(self, **kwargs):
        hidden_fields = set(
            attribute_name
            for attribute_name, model_field in self.__fields__.items()
            if model_field.field_info.extra.get("hidden") is True
        )
        kwargs.setdefault("exclude", hidden_fields)
        return super().dict(**kwargs)


class Book(CustomBaseModel):
    author: str
    title: str
    genre: str = Field(..., hidden=True)


m = Book(
    author="Antoine de Saint-Exupéry",
    title="The Little Prince",
    genre="Novella",
)
print(m.genre)  # Novella
print(m.dict())  # {'author': 'Antoine de Saint-Exupéry', 'title': 'The Little Prince'}

[trallnag]

Does anyone know what method / function FastAPI is using to generate the OpenAPI schema from Pydantic models? We could then remove the to be hidden fields on the fly like in @aberrier's example

It is not schema() or schema_json()

[sgiroux]

You need to modify the schema processing of Pydantic to achieve this. Easiest way I've found was to monkey the field_schema function and raise a SkipField error.

You can define your 'hidden' fields as follows:

from pydantic import BaseModel

class Model(BaseModel): 
    exposed_id: int = Field(1)
    hidden_id: int = Field(1, hidden_from_schema=True)`

Apply the patch

from pydantic import schema
def field_schema(field: ModelField, **kwargs: Any) -> Any:
    if field.field_info.extra.get("hidden_from_schema", False):
        raise schema.SkipField(f"{field.name} field is being hidden")
    else:
        return original_field_schema(field, **kwargs)

original_field_schema = schema.field_schema
schema.field_schema = field_schema

[botsman]

@sgiroux thanks!
I've used your solution in my project and it worked good!

I have some private fields which I want to provide for my "trusted" customers. Obviously, I don't want other people to see such fields in the documentation (because they will not get these fields in their responses anyway).

[Su57]

i have the same problem, but i can't find a elegant solution
let's say i have a schema used to create a user:

class UserCreationSchema(BaseModel):
    username: str
    password: str
    created_by: Optional[str] = None

but i don't want the created_by field exposed at the auto-genarated docs, in fact, the value of it should be generated by the backend.

the official doc mentioned above： add an output model just can't help, it's not the same thing.

so what's the best way to go about this?

[tonythree]

Hey @Su57 , I had exactly the same issue. Creating extra models (classes) can help.
In fact this is what FastAPI docs recommend (they use pydantic).
In your case,

class UserBase(BaseModel):
    username: str
    password: str

class UserCreationSchema(UserBase):
    created_by: Optional[str] = None

So the autogenerated docs can use UserBase class, while your backend can use UserCreationSchema class.
Hope this helps!

[a-recknagel]

You can also just make them private, fastapi respects that:

from pydantic import BaseModel, 
from datetime import datetime

class User(BaseModel):
    class Config:
        underscore_attrs_are_private = True

    name:str
    _create_at: datetime = Field(default_factory=datetime.now)

user = User(name="foo")

print(user._created_at)
# 2021-08-27 10:20:05.741129

print(user.dict())
# {'name': 'foo'}

print(User.schema())
# {
#     "title": "User",
#     "type": "object",
#     "properties": {"name": {"title": "Name", "type": "string"}},
#     "required": ["name"],
# }

[wookiesh]

You can also just make them private, fastapi respects that:

from pydantic import BaseModel, 
from datetime import datetime

class User(BaseModel):
    class Config:
        underscore_attrs_are_private = True

    name:str
    _create_at: datetime = Field(default_factory=datetime.now)

user = User(name="foo")

print(user._created_at)
# 2021-08-27 10:20:05.741129

print(user.dict())
# {'name': 'foo'}

print(User.schema())
# {
#     "title": "User",
#     "type": "object",
#     "properties": {"name": {"title": "Name", "type": "string"}},
#     "required": ["name"],
# }

That's probably the most elegant, but the "_" is not possible in my project, any way to mark the field as private as an attribute ?

Something along this:

class User(BaseModel):
    name: str = Field(private=True)

[ghandic]

Would it be ok in your project to make it private AND use an alias?

[a-recknagel]

@wookiesh As @ghandic said, the right way to handle that would be to assign like this:

class User(BaseModel):
    class Config:
        underscore_attrs_are_private = True

    _my_name: str = Field(alias="my_name")

Instantiation of that class will look like this User(my_name="Alice"), and the json keys that fastapi would accept/submit will also show that attribute only under it's alias.

Not exactly intuitive, but that's the way it is supported.

[lelecanfora]

this is broken in V2:

class BaseIdIn(BaseSchema):
    _id: UUID = Field(alias='id', default_factory=uuid4)

NameError: Fields must not use names with leading underscores; e.g., use 'id' instead of '_id'.

class BaseIdIn(BaseSchema):
    _id: UUID = PrivateAttr(alias='id', default_factory=uuid4)

TypeError: PrivateAttr() got an unexpected keyword argument 'alias'

[bauerem]

I'm having the same problem

[wookiesh]

Don't know why I was doubting, but yes indeed, that works in my case, thanks !

[gautamvij94]

Is there a way to conditionally remove fields tagged with (hidden=True) based on state in request ?
Eg.
If request.state.external=True then don't return certain fields defined in the response models as hidden=True

[Leon0824]

I am using a stupid way:

class UserDB(BaseModel):
    name: str
    password: str

class User(BaseModel):
    name: str

@app.get('/user/{name}', response_model=User)
def create_user(name: str):
    user_db = db.select('user', 'name', name)
    user = User(name=user_db.name)
    return user

Just do not inhererit from its brother model.

[arkanmgerges]

Maybe this will help, by using SecretStr with Field and exclude=True

from typing import Union

from fastapi import FastAPI
from pydantic import BaseModel, EmailStr, SecretStr

app = FastAPI()
@app.post("/user/", response_model=UserOut)
async def create_user(user: UserIn):
    return user

class UserIn(BaseModel):
    username: str
    password: str
    email: EmailStr
    full_name: Union[str, None] = None

class UserOut(UserIn):
    password: SecretStr = Field(..., exclude=True)

[vahidzhe]

This working,thanks)

[Xenthys]

I'm late to the party, but if you want to hide Pydantic fields from the OpenAPI schema definition without either adding underscores (annoying when paired with SQLAlchemy) or overriding the schema.field_schema function that will display warnings in your logs, you can customize the schema according to Pydantic's documentation.

You can therefore add a schema_extra static method in your class configuration to look for a hidden boolean field option, and remove it while still retaining all the features you need.

Here is how I did it:

from pydantic import BaseModel, Field

class User(BaseModel):
    public_field: str
    hidden_field: str = Field(hidden=True)

    class Config:
        @staticmethod
        def schema_extra(schema: dict, _):
            props = {}
            for k, v in schema.get('properties', {}).items():
                if not v.get("hidden", False):
                    props[k] = v
            schema["properties"] = props

This can also be done by creating a "proxy" BaseModel class per example.
All your existing classes will therefore obtain this feature without editing them:

from pydantic import BaseModel as _BaseModel

class BaseModel(_BaseModel):
    class Config:
        @staticmethod
        def schema_extra(schema: dict, _):
            props = {}
            for k, v in schema.get('properties', {}).items():
                if not v.get("hidden", False):
                    props[k] = v
            schema["properties"] = props

You can certainly optimize this function, but that's the cleanest way I've been able to find.

[lelecanfora]

any idea how to make it work with V2 and the model_config attribute?

[fbaldi6]

I think this would work

from pydantic import BaseModel, ConfigDict

class MyModel(BaseModel):
      @staticmethod
      def schema_extra(schema: dict, _):
          props = {}
          for k, v in schema.get('properties', {}).items():
              if not v.get("hidden", False):
                  props[k] = v
          schema["properties"] = props

      model_config = ConfigDict(json_schema_extra=schema_extra)

[manasrb21]

I think you could also do SkipJsonSchema :

from pydantic.json_schema import SkipJsonSchema
from pydantic import Field
from typing import Optional

class User(BaseModel):
    public_field: str = Field(..., alias="public_field")
    hidden_field: SkipJsonSchema[Optional[str]] = Field(None, alias="hidden_field")

P.S I am using pydantic==2.6.1
https://docs.pydantic.dev/latest/api/json_schema/#pydantic.json_schema.SkipJsonSchema

[Ae-Mc]

Well, yes, but actually no:

from fastapi import ApiRouter
from pydantic import BaseModel, Field
from pydantic.json_schema import SkipJsonSchema


router = ApiRouter()


class TestModel(BaseModel):
    hidden: SkipJsonSchema[int] = Field(0)
    shown: int = Field(1)


@router.post("/test")
async def test_func(model: TestModel = Depends()):
    pass

results in

[afikbar]

@Ae-Mc did you manage to solve it? It appears that SkipJsonSchema isnt working when using Depends() on the pydantic class. using pydantic 2.7.1 \ fastapi 0.111.0

[tux-mind]

import pydantic

class A(pydantic.BaseModel):
    name: str
    surname: str

class B(A):
    pass

if type(B.__fields_set__) == set:
    B.__fields_set__.remove('surname')
del B.__fields__['surname']

print(B.schema_json())

[Delnold]

I found an interesting way of making it, although I'm not sure how reliable it is.

from pydantic import root_validator
from pydantic import BaseModel

class RecipeCreate(BaseModel):
    name: str
    description: str
    instructions: str
    prep_time: int
    cook_time: int
    servings: str

    @root_validator
    def calculate_total_time(cls, values):
        values['total_time'] = values['prep_time'] + values['cook_time']
        return values

class RecipeView(BaseModel):
    name: str
    description: str
    instructions: str
    prep_time: int
    cook_time: int
    servings: str
    total_time: int

[a-recknagel]

That pattern is also part of the orm documentation of fastapi, with the idea being "how to expose a data-interface to users without secrets/internal values that generated without them setting them" https://fastapi.tiangolo.com/tutorial/sql-databases/?h=#create-the-pydantic-models

You can reduce code duplication by declaring and inheriting from base-classes.

[lucasgadams]

What is the best way to deal with conditionally hiding fields that are deeply nested? The two approaches I see would be to explicitly exclude it in the serialization methods, or to create a whole mess of different class variants. Here is an example:

from pydantic import BaseModel


# Models with all fields
class Chunk(BaseModel):
    text: str
    embedding: list[float]


class Document(BaseModel):
    id: str
    name: str
    chunks: list[Chunk]


class User(BaseModel):
    id: str
    name: str
    documents: list[Document]


class Team(BaseModel):
    id: str
    name: str
    users: list[User]


class Company(BaseModel):
    id: str
    name: str
    teams: list[Team]


# Reponse models with hidden fields. The only field we actually need to hide is "embedding", deeply nested
class ChunkResponse(BaseModel):
    text: str
    # embedding field is now hidden


class DocumentResponse(BaseModel):
    id: str
    name: str
    chunks: list[ChunkResponse]


class UserResponse(BaseModel):
    id: str
    name: str
    documents: list[DocumentResponse]


class TeamResponse(BaseModel):
    id: str
    name: str
    users: list[UserResponse]


class CompanyResponse(BaseModel):
    id: str
    name: str
    teams: list[TeamResponse]



# CompanyResponse is returned from a `get_company` endpoint, and should have all fields except for embedding.


# Seemingly we need to create a new "Response" model for every single nested class just to hide the single field.
# Every single model is identical to the original except except for the Chunk model

Is there a better way to do this? Seems like a crazy amount of code duplication just to conditionally hide the embedding field. For sure I could implement some custom serialization with the exclude flag, but that wont have good typing and could break easily. Ideally I could have some flag set on the embedding Field that would just say "dont emit this under X scenario". Am I thinking about this wrong?

[osangu]

@Su57 @tonythree @a-recknagel @lelecanfora

I think i found most elegance way to hide field!!

from datetime import datetime

from fastapi import FastAPI

from pydantic import BaseModel, PrivateAttr, computed_field

app = FastAPI()


class CreateUserRequest(BaseModel):
    email: str
    _created_at: int = PrivateAttr(default_factory=datetime.now)

    @computed_field(
        # repr=False,
        return_type=datetime
    )
    def created_at(self):
        return self._created_at


@app.post('/')
def haha(request: CreateUserRequest):
    return request.model_dump()

By using PrivateAttr and computed_field, It didn't show at swagger.
But when execute model_dump the field came out!!
isn't it cool?

[seanhuang514]

I use SkipJsonSchema to hide field in pydantic v2, You can check the link to get more information

Add this as an annotation on a field to skip generating a JSON schema for that field.

from pydantic.json_schema import SkipJsonSchema

class User(BaseModel):
    username: str
    password: str

class UserResponseSchema(User):
    password: SkipJsonSchema[str]

[visinfo]

I defined my request model like below and it got removed from FastAPI docs .

from pydantic import BaseModel,  PrivateAttr

class VersionRequest(BaseModel):
         _version: int = PrivateAttr(default=1)

further more I defined the methods to set and get the version value so get rid of compiler error Access to a protected member _version of a class