2019-03-21: THIS PROJECT IS NO LONGER UNDER ACTIVE DEVELOPMENT
Have a look at cf-butler
This is a Spring Boot application that employs the Reactive support in both the Pivotal Application Service Java Client and your choice of either Spring Boot Starter Data Mongodb or rxjava2-jdbc with an HSQL backend. These libraries are employed to generate custom application inventory detail and summary reports from a target foundation. It may optionally be configured to send an email to recipient(s) with those reports attached.
While the cf-app-inventory-report does not take the place of an official foundation Accounting Report, it does provide a much more detailed snapshot of all the applications that were currently running at the time of collection. The Accounting Report is focussed on calculating aggregates (on a monthly basis) such as: (a) the total hours of application instance usage and (b) the largest # of application instances running (a.k.a. maximum concurrent application instances).
Required
- Pivotal Application Service account
Optional
- Email account
- SMTP Host
- SendGrid account
git clone https://github.com/pacphi/cf-app-inventory-report.git
Make a copy of then edit the contents of the application.yml
file located in src/main/resources
. A best practice is to append a suffix representating the target deployment environment (e.g., application-pws.yml
, application-pcfone.yml
). You will need to provide administrator credentials to Apps Manager for the foundation if you want to get a complete inventory of applications.
You really should not bundle configuration with the application. To take some of the sting away, you might consider externalizing and/or encrypting this configuration.
Place secrets in config/secrets.json
, e.g.,
{
"CF_API-HOST": "xxxxx",
"CF_USERNAME": "xxxxx",
"CF_PASSWORD": "xxxxx",
"MAIL_FROM": "xxxxx",
"MAIL_RECIPIENTS": "xxxxx",
"SENDGRID_API-KEY": "xxxxx"
}
We'll use this file later as input configuration for the creation of either a credhub or user-provided service instance.
Replace occurrences of
xxxxx
above with appropriate values
At a minimum you should supply values for the following keys
cf.apiHost
- a Pivotal Application Service API endpointtoken.provider
- Authorization token provider, options are:userpass
orsso
Based on choice the authorization token provider
cf.username
- a Pivotal Application Service account username (typically an administrator account)cf.password
- a Pivotal Application Service account password
cf.refreshToken
- the refresh token to be found within~/.cf/config.json
after your authenticate
notification.engine
- email provider, options are:none
,java-mail
orsendgrid
If you set the email provider to
none
, then no email will be delivered
spring.mail.host
- an SMTP hostspring.mail.username
- an email account usernamespring.mail.password
- an email account passwordmail.from
- originator email addressmail.recipients
- email addresses that will be sent an email with CSV attachments
spring.sendgrid.api-key
- an api key for your SendGrid accountmail.from
- originator email addressmail.recipients
- email addresses that will be sent an email with CSV attachments
If you edited the contents of application.yml
then you could set spring.profiles.active
to one of either mongo
or jdbc
.
E.g., you could start the app with an HSQL backend using
./gradlew bootRun -Dspring.profiles.active=jdbc
If you copied and appended a suffix to the original application.yml
then you would set spring.profiles.active
to be that suffix
E.g., if you had a configuration file named application-pws.yml
./gradlew bootRun -Dspring.profiles.active=pws
See the samples directory for some examples of configuration when deploying to Pivotal Web Services or PCFOne.
On application start-up, a versioned Mongo executable is downloaded from a default location (addressable from the public internet). If you would like to download the executable from an alternate location and/or select an alternate version, add the following:
spring.mongodb.embedded.verson
- version of the Mongo executable (e.g.,3.4.15
)spring.mongodb.embedded.download.path
- the path to the parent directory hosting OS-specific sub-directories and version(s) of Mongo executables (e.g.,https://fastdl.mongodb.org/
)spring.mongodb.embedded.download.alternate
- this is a boolean property and must be set to true to activate alternate download URL
As an example, the following
spring:
mongodb:
embedded:
version: 3.4.15
download:
path: https://fastdl.mongodb.org/
alternate: true
would download the Mongo executable from https://fastdl.mongodb.org/osx/mongodb-osx-x86_64-3.4.15.tgz
when the app is running on a Mac OSX host.
OS-specific sub-directory choices are:
linux
,win32
, andosx
. See https://www.mongodb.com/download-center#community for more details.
Update the value of the cron
property in application.yml
. Consult this article and the Javadoc to understand how to tune it for your purposes.
To have access to a database management console which would allow you to execute queries against the in-memory database, you will need to set an additional JVM argument.
-Djava.awt.headless=false
Note: this is not an available option when deploying to a PAS foundation.
./gradlew build
./gradlew bootRun -Dspring.profiles.active={backend_provider}
where {backend_provider}
is either mongo
or jdbc
You'll need to manually stop to the application with
Ctrl+C
The following instructions explain how to get started when token.provider
is set to userpass
Authenticate to a foundation using the API endpoint.
E.g., login to Pivotal Web Services
cf login -a https://api.run.pivotal.io
The following instructions explain how to get started when token.provider
is set to sso
Authenticate to a foundation using the API endpoint
E.g., login to PCF One
cf login -a https://api.run.pcfone.io -sso
Visit the link in the password prompt to retrieve a temporary passcode, then complete the login process
E.g.,
https://login.run.pcfone.io/passcode
)
Inspect the contents of ~/.cf/config.json
and copy the value of RefreshToken
.
Paste the value as the value for CF_REFRESH-TOKEN
in your config/secrets.json
{
"TOKEN_PROVIDER": "sso",
"CF_API-HOST": "xxxxx",
"CF_REFRESH-TOKEN": "xxxxx",
}
Deploy the app (w/ a user-provided service instance vending secrets)
./deploy.sh
Deploy the app (w/ a Credhub service instance vending secrets)
./deploy.sh --with-credhub
Shutdown and destroy the app and service instances
./destroy.sh
Utilizes cf CLI to query foundation for application details across all organizations and spaces for which the account is authorized. Generates an email with a couple of attachments, then sends a copy to each recipient.
Sample
PCF Application Inventory Report
Sample
Please find attached application inventory detail and summary reports from api.run.pivotal.io generated 2018-05-30T10:55:08.247.
Sample app-inventory-detail.csv
organization,space,application name,buildpack,image,stack,running instances,total instances,urls,last pushed,last event,last event actor,requested state
Northwest,cphillipson,cloud-native-spring-ui,,java,cflinuxfs2,0,1,cloud-native-spring-ui-riskiest-ghee.cfapps.io,2017-12-13T15:56:26,,,stopped
Northwest,sdeeg,nodefire-smd,nodejs,,cflinuxfs2,1,1,nodefire-smd.cfapps.io,2018-05-03T17:14:10,,,started
Northwest,cphillipson,cloud-native-spring,java,,cflinuxfs2,0,1,cloud-native-spring-laboured-estragon.cfapps.io,2017-12-13T14:54:18,,,stopped
zoo-labs,test,cphillipson-ruby-demo,ruby,,cflinuxfs2,1,1,cphillipson-ruby-demo.cfapps.io,2018-05-29T13:25:36,audit.app.droplet.create,[email protected],started
zoo-labs,test,cook,java,,cflinuxfs2,0,1,cook-impressive-bonobo.cfapps.io,2018-05-10T16:48:18,audit.app.stop,[email protected],stopped
Northwest,nthomson,bookstore-connector,java,,cflinuxfs2,1,1,bookstore-connector.cfapps.io,2018-04-09T16:40:23,,,started
Northwest,nthomson,bookstore-service-broker,java,,cflinuxfs2,1,1,bookstore-service-broker.cfapps.io,2018-04-09T12:49:41,,,started
...
Sample app-inventory-summary.csv
organization,total
Northwest,32
zoo-labs,4
buildpack,total
java,25
unknown,5
staticfile,2
dotnet,2
ruby,1
nodejs,1
last pushed,application total
<= 1 day,2
> 1 day <= 1 week,11
> 1 week <= 1 month,6
> 1 month <= 3 months,3
> 3 months <= 6 months,6
> 6 months <= 1 year,8
> 1 year,0
state,instance total
started,14
stopped,30
all,44
Total applications: 36
For additional convenenience REST endpoints have been exposed for on-demand reporting. Report results are refreshed on the cron
schedule mentioned above.
GET /report
Produces
text/plain
output combining detail and summary application info
GET /users
Produces
application/json
output containing user/role details for all orgs/spaces.
GET /users/{organization}/{space}
Produces
application/json
output containing user/role details for one org/space.
Tip of the hat to those who've gone before...