Skip to content

Latest commit

 

History

History
372 lines (238 loc) · 19 KB

README.md

File metadata and controls

372 lines (238 loc) · 19 KB

Synthesizer API 🎹

🔥Synthesizer API Explorer!🔥
🔥Synthesizer Directory🔥 🔥Synthesizer Directory repo🔥

For all you gearslutz out there! This free API is the only one out there (since I checked). Explore over 800 synthesizers from past and present. Get insight in their manufacturers and specifications. The API builds upon the data available on the vintagesynthexplorer website. Future work will be aimed to broaden querying options and let you fellow synth lovers contribute to the collection.


Growing as a Developer:

For Usage:


About this Project


The goal of this project is to build an public REST API 🎯 and thereby applying existing knowledge and explore new terrains. This was done independently by reading documentation, watching online videos, consulting StackExchange & sometime my coach.

Although this project aimed at making a public API, I obviously tried it out myself! Therefore I made a frontend that uses it. In development some features where added to the backend as well. Such as a route to suggest synthesizer and an admin login system.


Be sure to visit this deployed version as well: 🔥Synthesizer Directory🔥

This repo contains three main folders:


SCRAPER

The scraper folder contains DOM oriented programming. Before I could fill the database with synthesizers, manufacturers & specification I needed to scrape data. My source also had images, therefore I needed to download images from their respective URL's. After scraping the data it needed to be normalized to fit the SQL database structure. For this, some JS object/string manipulation was used.

SERVER

This folder is the heart of the project. It contains the server (with database). You will find endpoints (& queries) making use of validation middlewares (& Yup). People are able to suggest synths that can get approved through an admin login-system. Look around in authorization (& authorization in model) done through Express middlewares with JWT tokens. The development was mostly fuelled by end-to-end testing. Another thing that needed to be done was migrating images to Cloudinary db and save the image URL's to the main SQL database.

CLIENT

Here you can find the Synthesizer API explorer repo, done with ReactApp. Through the explorer an apikey can be sent to your email. It utilizes frontend validation(with React Hook Form) and some storage concepts(useContext) for the apikey. :fire:Go to API explorer!🔥



Leading topics:


ENVIRONMENTS

Throughout the project we have used: development, testing and production environments. Switching in/out sources and storing key's and secret's in .env and hosts. Wrapping my head around it was challenging at first. Now I feel comfortable to apply this in real world situations.


END-TO-END TESTING

All end-point have been thoroughly tested. The testing formed the basis of development Checking if correct responses were sent & testing all possible 'un-happy' paths was an endevour. But it pays out by making me feel safe to refactor code. Jest and Supertest were used in de backend, Jest & Playwright (like Puppeteer) in the frontend.


VALIDATION & ERROR HANDLING

Validation was done with Yup, used in the backend as well as the frontend. Together with middlewares to safeguarded the routes from user errors and send meaningful feedback to the client when they do occur. Also validated in the backend: multipart/form-data & attachments.


AUTHENTIFICATION & PASSWORD HASHING

To be able to control the flow of user suggested synthesizers I implemented a admin login-system with JWT token. Although this was not my first encounter with this, it was the first time that I implemented it inside of my model and validated the expiration of the token. Hashing/comparing of the user's password was another thing that was done, also inside the model directly.


Things I've done in the backend:

DOM manipulation downloading images ⭐ managing multiple dsb’s normalizing data
generating tables relating tables mocking seed data ⭐ using environments ⭐
managing secrets deploying writing npm scripts big O notation ⭐
end-to-end testing ⭐ pagination validation sending emails ⭐
writing middlewares routing middlewares continuous Integration ⭐ error handling
CORS handling FormData HTTP methods (all) Netlify
Heroku JSDOM Yup ⭐ Express
Cloudinary (db) ⭐ Sendgrid ⭐ Jest Supertest ⭐
Sequelize ORM ElephantSQL SQL JWT
custom Promises Express Node.js


Things I've done in the frontend:

React DOM routing Bootstrap ⭐ Sass caching ⭐
copy to clipboard custom Hooks UseContext ⭐ local storage
React children ⭐ validation error handling fetching data

New technologies learned in this project


User Stories

  • As a Developer that uses this API I want to be able to fetch manufacturers and synthesizer with their specifications, so I can use the API where it is meant for

  • As a Developer that uses this API I want to be able to fetch images of the synthesizers, so I can make something visual I my frontend

  • As a Developer that uses this API I want to be able to know what mistakes I made when I did something wrong in fetching the data,so I can adjust my query and still find the data I'm looking for

  • As a Developer that uses this API I want to be able to paginate and limit the results I'm getting, so I can easily built this in my frontend

  • As a Developer that uses this API I want to be able to query a specific synthesizer either by id or by name, so I'm not restricted to using two endpoint that share one purpose

  • As a Developer that uses this API I want to get the results sorted by year that the synthesizers where produced, so I can easily implement this behaviour when building the frontend

  • As a Developer that uses this API Explorer I want to be able to request apikey through email, so I can explore this API

  • As a Developer that uses this API Explorer I want to be able to view the JSON data retrieve from the URL, so I will see the data that is available

  • As a Developer that uses this API Explorer I want to be able to easily create the URL with route, params and query, so I will find the data that is available

  • As a Developer that uses this API Explorer I want to get suggestions of the endpoint, params and queries I'm aloud to enter, so I know what routes, params and queries are available

  • As a Developer that uses this API Explorer I want to be able to store the apikey across sessions, so I don't have to bother checking my email to find it

  • As a Developer that uses this API Explorer I want to copy the explorer URL to my clipboard so, I can quickly use it outside of the Synthesizer API Explorer

the following stories are derived form the Synthesizer Deirectory frontend build upon this API at later stage

  • As a User of the Synthesizer Directory I want to be able to suggest new synthesizers, so I can contribute to the collection

  • As Myself I want to be able to judge the suggested synthesizer, so I can decide which one's will be added to the collection


Project Board

  • You can find it here.

Wireframes API Explorer


Datamodel

  • Checkout the datamodel used as base for data structure.

Git Version Control

...





Usage


API key

For each request send, you have to send your API key with it. It can be obtained through registering your email at API explorer. The API key can be located anywhere inside the query string. The following example requests with and without additional queries are all valid:

.../synths?key=<apiKey>
.../synths?yearProduced=1980&key=<apiKey>
.../synths?key=<apiKey>&yearProduced=1980

Pagination


Calling any API endpoint without a resource ID or name will return a paginated list of available resources for that API. By default, a "page" will contain up to 20 resources. If you would like to change this just add a 'limit' query parameter to the GET request, e.g. ?=60. You can use 'offset' to move to the next page, e.g. ?limit=60&offset=60.

Name Result Type
offset Get all synths/manufacturers with offset integer
limit Get all synths/manufacturers with limit integer


Endpoint: Synthesizers


GET /synths

Get a list of synth resources including their image, manufacturers and ten specifications

NOTE:
Some data is not normalized yet. Some specifications will have a null value.

Example of json resources:

{
    "count": 815,
    "synths": [
        {
            "id": 309,
            "name": "Native Instruments Pro-53",
            "img": "https://res.cloudinary.com/dgte68hwi/image/upload/v1613044108/Native_Instruments_Pro-53.jpg",
            "ManufacturerId": 50,
            "SpecificationId": 309,
            "synthType": null,
            "interfaceFeature": null,
            "format": null,
            "Specification": {
                "id": 309,
                "polyphony": "32 voices per Instance",
                "keyboard": null,
                "control": null,
                "yearProduced": 2002,
                "memory": "512 Programs (8 programs per bank, 8 banks per file, 8 files). Unlimited Files (512 programs) can be created and stored on your computer for virtually unlimited memory.",
                "oscillators": "2 Oscillators. Osc. A has sawtooth and pulse waveforms. Osc. B has sawtooth, triangle and pulse waveforms.",
                "filter": "24 dB/oct low-pass / high-pass filter with resonance, self oscillation, ADSR envelope",
                "lfo": "1 with sawtooth, square and pulse waveforms, MIDI sync, envelope triggering. Assignable to oscillators, filter, etc.",
                "effects": "Delay Section creates Multi-Echo, Chorus, Flange effects and is MIDI-syncable"
            },
            "Manufacturer": {
                "id": 50,
                "manufacturer": "Native Instruments"
            }
        },

GET /synths/1

GET /synths/Jupiter-8

Get one synth resource, by either giving it's ID or exact name.

NOTE:
All specifications and the manufacturer are of synth resources are delivered in a request by default. Specifying them specifically in a query is not necessary. Some data is not normalized yet. This makes it only possible to query with exact name matches. Look into future work to see development goals for querying.

Name Result Type
manufacturer Get all synths by a given manufacturer string
yearProduced Get all synths by a given year they hit the market integer
polyphony Get all synths with description of amount of voices string
keyboard Get all synths with description of keyboard specifics string
control Get all synths with description of given control (e.g. MIDI, CV/GATE) string
memory Get all synth with description of memory string
oscillators Get all synths with description of oscillators string
filter Get all synths with description of filter string
lfo Get all synths with description of lfo string
effects Get all synths with description of effects string


Endpoint: Manufacturers


GET /manufacturers

Get a list of manufacturer resources.

GET /manufacturers/1

GET /manufacturers/Roland

Get one manufacturer resource, by either giving it's ID or exact name.




Future Work


This API is a work in process. Future work aims to improve querying, removing and/or adding queriable synth specifications and creating a moderated POST endpoint, to make you able to add to this collection.

PLANNED

Name Result Type Options
manufacturer Get all synths produced by a given manufacturer string name or id
yearProduced Get all synths that hit the market by a given year integer range, asc/dec
polyphony Get all synths that have a given amount of voices integer range, asc/dec
keyboard Get all synths that have or do not have a keyboard ENUM none/yes
control Get all synths that are controlled with MIDI and/or CV/GATE ENUM MIDI, CV/GATE
oscillators Get all synths that have a given amount of oscillators integer range, asc/dec
oscillatorType Get all synths that are either VCO or DCO ENUM VCO, DCO
filter Get all synths that contain one or a combination of lowpass, highpass, bandpass ENUM lowpass, highpass, bandpass
effects Get all synths that have or do not have built in effects ENUM none/yes
type Get all synths that are either analog, digital or hybrid ENUM analog, digital, hybrid


Updates


  • Adding sort by year, ascending and descending
  • Adding, merging specification that will be queriable
  • Make querying possible without strictly needing to equal the resource name
  • Sorting sorting by number functionalities used in more than yearProduced alone
  • And there will be more...