- Plugin for collecting cost data from CSV file
ㅡ
This plugin is a plugin that collects cost data from a CSV file.
The CSV file must be in the format specified in the 2) CSV format section.
The CSV file must be located on the web server,
and the URL must be specified in the 3) options of the plugin.
If you have completed understanding the steps 2) and 3),
you can check the actual usage in 4) How to use.
- The above is an example of a csv file, and the fields that exist in the csv must exist.
- Here is a list of available fields. Of these,
cost
fields are required field. - The required date fields are:
- If there is a
billed_date
(ex. "2023-09-01") field, it works even if there are noyear
andmonth
fields. - This works if there is no
billed_date
field and there areyear
,month
, andday
fields. - If there is no
billed_date
field and there areyear
andmonth
,day
is applied as 1 day. - Does not work except for the above cases.
- If there is a
- cost (required)
- billed_date (required)
- year (required)
- month (required)
- day (optional)
- usage_quantity
- usage_type
- provider
- region_code
- product
- The following options are available for the plugin.
- The options are specified in the form of a YAML file.
- You can find out how yaml is used in 4) How to use section.
- An example using all options is shown below.
# update_data_source_options.yml --- options: base_url: - https://raw.githubusercontent.com/cloudforet-io/plugin-http-file-cost-datasource/master/examples/cost_example.csv - https://raw.githubusercontent.com/cloudforet-io/plugin-http-file-cost-datasource/master/examples/custom_cost_example.csv - https://raw.githubusercontent.com/cloudforet-io/plugin-http-file-cost-datasource/master/examples/example_with_billed_at.csv - https://raw.githubusercontent.com/cloudforet-io/plugin-http-file-cost-datasource/master/examples/examples_of_different_headers.csv field_mapper: cost: TotalCost currency: CurrencyCode day: Day month: Month provider: Provider usage_quantity: UsageQuantity usage_type: UsageType year: Year default_vars: currency: KRW billed_at: UsageStartDate
base_url (required)
- The URL of the CSV file to be used.
- The URL must be a list type.
---
options:
base_url:
- https://raw.githubusercontent.com/cloudforet-io/plugin-http-file-cost-datasource/master/examples/cost_example.csv
- https://raw.githubusercontent.com/cloudforet-io/plugin-http-file-cost-datasource/master/examples/custom_cost_example.csv
- https://raw.githubusercontent.com/cloudforet-io/plugin-http-file-cost-datasource/master/examples/example_with_billed_at.csv
- https://raw.githubusercontent.com/cloudforet-io/plugin-http-file-cost-datasource/master/examples/examples_of_different_headers.csv
field_mapper (optional)
- The field name of the CSV file to be used.
- If you do not specify this option, the default column is used.
- As you can see in examples_of_different_headers.csv, if the column names are different, you can use these options to map them.
---
options:
field_mapper:
cost: TotalCost
currency: CurrencyCode
day: Day
month: Month
provider: Provider
usage_quantity: UsageQuantity
usage_type: UsageType
year: Year
default_vars (optional)
- If you want to set a default value for a field, you can use this option.
- If you do not specify this option, the default column is used.
- As you can see in example_of_default_vars.csv, This can be used if you want to default all currencies to KRW.
---
options:
default_vars:
currency: KRW
In order to use the plugin, how to use spacectl CLI tools must be preceded.
- Check if the plugin you want to use from the marketplace exists.
$ spacectl list repository.Plugin -p service_type=cost_analysis.DataSource --minimal
plugin_id | name | image | state | service_type | registry_type
----------------------------------+-------------------------------------+-------------------------------------------+---------+--------------------------+-----------------
plugin-http-file-cost-datasource | HTTP file Cost Analysis Data Source | pyengine/plugin-http-file-cost-datasource | ENABLED | cost_analysis.DataSource | DOCKER_HUB
- Register with the DataSource resource of cost-analysis.
$ spacectl exec register cost-analysis.DataSource -f register_data_source.yml
# register_data_source.yml
---
name: HTTP File Data Source
service_type: EXTERNAL
image: pyengine/plugin-http-file-cost-datasource
tags: { }
template: { }
- Check the registered CSV Plugin information.
$ spacectl exec get cost-analysis.DataSource -p data_source_id=<data_source_id>
---
created_at: '2023-02-06T11:04:34.348Z'
data_source_id: ds-123456789012
data_source_type: EXTERNAL
domain_id: domain-123456789012
last_synchronized_at: '2023-02-06T16:00:08.356Z'
name: HTTP File Data Source
plugin_info:
metadata:
data_source_rules:
- actions:
match_service_account:
source: account
target: data.account
conditions: []
conditions_policy: ALWAYS
name: match_service_account
options:
stop_processing: true
tags: {}
plugin_id: plugin-http-file-cost-datasource
upgrade_mode: AUTO
version: 1.0.0.20230206.225536
state: ENABLED
tags: {}
template: {}
- Sets the options corresponding to the url where the csv file is located in the plugin.
$ spacectl exec update_plugin cost-analysis.DataSource -p data_source_id=<data_source_id> -f update_data_source_options.yml
# update_data_source_options.yml
---
options:
base_url:
- https://raw.githubusercontent.com/cloudforet-io/plugin-http-file-cost-datasource/master/examples/cost_example.csv
- https://raw.githubusercontent.com/cloudforet-io/plugin-http-file-cost-datasource/master/examples/custom_cost_example.csv
- https://raw.githubusercontent.com/cloudforet-io/plugin-http-file-cost-datasource/master/examples/example_with_billed_at.csv
- https://raw.githubusercontent.com/cloudforet-io/plugin-http-file-cost-datasource/master/examples/examples_of_different_headers.csv
field_mapper:
cost: TotalCost
currency: CurrencyCode
day: Day
month: Month
provider: Provider
usage_quantity: UsageQuantity
usage_type: UsageType
year: Year
default_vars:
currency: KRW
billed_at: UsageStartDate
- Manually sync the cost information of the csv file in step 4.
$ spacectl exec sync cost-analysis.DataSource -p data_source_id=<data_source_id>