Create Your Terraform Provider from OpenAPI / Swagger

Introduction

Terraform is a key infrastructure-as-code tool that uses providers to manage cloud infrastructure via API calls. Creating and maintaining these providers, typically written in Go, can be challenging due to the need for specialized skills and continuous updates with API changes. Speakeasy offers a streamlined solution by allowing the generation of Terraform providers directly from OpenAPI specifications. This approach minimizes the need for Go expertise and ensures providers are consistently updated, simplifying the development and maintenance of Terraform providers for complex cloud environments.

What does Speakeasy generate

Speakeasy generates a fully-functional Terraform Provider from an annotated (opens in a new tab) OpenAPI Specification. The platform is able to create every element specified above to create a fully functional provider without any need to write go code. Speakeasy generates a provider using the terraform-plugin-framework (opens in a new tab).

Provider Components
Provider ComponentsSpeakeasy generation
Resource SchemasGenerated by adding the x-speakeasy-entity-operation: MyEntity#create to any operation in the OpenAPI spec. Hoists and merges all JSON Schemas associated with all create/read/update/delete operations appropriately.
Data Source SchemasGenerated by adding the x-speakeasy-entity-operation: MyEntity#read to any operation in the OpenAPI spec. Hoists and merges all JSON Schemas associated with all read operations into a single data source.
Create MethodsGenerated by adding the x-speakeasy-entity-operation: MyEntity#create to any operation in the OpenAPI spec. Speakeasy generates platform connector logic to invoke all API requests, and save responses into terraform state.
Read MethodsGenerated by adding the x-speakeasy-entity-operation: MyEntity#read to any operation in the OpenAPI spec. Speakeasy generates platform connector logic to invoke all API requests, and save responses into terraform state. These operations are also invoked and connected to state in CREATE/UPDATE methods should some data attributes only be available by making an additional API call. By defining these, drift detection and import is enabled on your terraform provider for all associated API response attributes.
Update MethodsGenerated by adding the x-speakeasy-entity-operation: MyEntity#update to any operation in the OpenAPI spec. Speakeasy generates platform connector logic to invoke all API requests, and save responses into terraform state. All attributes available in a create request but unavailable in an update request are associated with a plan modifier that forces a resource recreation when changed.
Delete MethodsGenerated by adding the x-speakeasy-entity-operation: MyEntity#delete to any operation with all non-optional API request properties available in the OpenAPI spec. Contains connector
Plan ValidatorsGenerated when using restricted OpenAPI data types (e.g. JSON fields, date fields, date-time fields) as will as OpenAPI validations. Also generated for some specific Speakeasy extensions when a customer requests it (e.g. x-speakeasy-conflicts-with)
Plan ModifiersGenerated when needed to ensure API and terraform state/plan have the appropriate semantics, for instance to work around diff-detection issues.
Resource ImportsGenerated by adding the x-speakeasy-entity-operation: MyEntity#read to an operation in the OpenAPI spec which has a singular ID field.

What does Speakeasy Support

Speakeasy follows OpenAPI semantics when generating a provider. In most simple CRUD APIs, very few annotations are required on your OpenAPI specifications, with all of the following inferred from the specification.

Supported OpenAPI Semantics
OpenAPI SemanticsSpeakeasy Support / Comment
Resource Schemas (const)When an attribute in an OpenAPI specification is specified as const, it is removed from a terraform schema and always sent in the request/assumed in the response
Resource Schemas (default)When an attribute in an OpenAPI specification is specified with a default, the resource will have this attribute Default to this value. If it is unset in a terraform specification, we will assume it is set to this value.
Server ConfigurationA server_url variable is available in the provider to enable it to be invoked against any API server. However this is defaulted to the first entry in the servers field of your OpenAPI specification
Global AuthenticationEvery authentication mechanism which relies on static authorization is supported: with their values automatically available to be configured in the provider configuration. For oAuth, a custom hook will need to be written container your authentication flow logic.
Query Parameter Serialization✅️Full support. All query parameter attributes will be available as resource/data source attributes in a terraform-native form. Might require remapping to through x-speakeasy-match.
Request Headers✅️Full support. All request attributes will be available as resource/data source attributes in a terraform-native form.
Multiple API requests in one CRUD step (e.g. "create" requiring 2 API calls)Full Support
JSON Schema type: stringFull Support
JSON Schema type: numberFull Support
JSON Schema type: integerFull Support
JSON Schema type: booleanFull Support
JSON Schema type: objectFull Support
JSON Schema type: nullFull Support
JSON Schema required: [requiredPropertyA, ...]Full Support. Combined into Required or Optional terraform attribute modifiers.
JSON Schema enum: [...values...]Full Support. A plan validator is added to assert only one of any of the pre-defined values are set.
JSON Schema type: arrayFull Support.
JSON Schema type: array, minItems: NFull Support. A plan validator is added to ensure the terraform ListAttribute has N items set.
JSON Schema type: array, maxItems: JFull Support. A plan validator is added to ensure the terraform ListAttribute has J items set.
JSON Schema type: array, uniqueItems: trueFull Support. A plan validator is added to ensure the terraform ListAttribute has all items as unique values.
JSON Schema type: number, format: floatFull Support.
JSON Schema type: number, format: doubleFull Support.
JSON Schema type: integer, format: int32Full Support.
JSON Schema type: integer, format: int64Full Support.
JSON Schema type: string, format: dateFull Support. A plan validator is added to ensure that a YYYY-MM-DD formatted value is set in this String value. E.g. "2022-01-30"
JSON Schema type: string, format: date-timeFull Support. A plan validator is added to ensure that a RFC3339 compatible value is set in this String value. E.g. "2019-10-12T07:20:50.52Z"
JSON Schema format: binaryFull Support. Accessable as a String attribute
JSON Schema nullable: trueFull Support. Combined into Required or Optional terraform attribute modifiers.
JSON Schema additionalProperties: trueFull Support. Note that a free-form object (without additional properties: true) is treated as an empty object
JSON Schema additionalProperties: ${JSON Schema}Full Support.
JSON Schema oneOf: [${JSON Schema}, ...]Full Support. Represented as a nested object with a child attribute per oneOf subschema. A plan validator is added that asserts only one child attribute can be set.
JSON Schema anyOf: [${JSON Schema}, ...]Full Support. Considered the same as oneOf
JSON Schema allOf: [${JSON Schema}, ...]Full Support. Constructs an "uber-type" from merging together the superset of all subschemas
JSON Schema "Any" TypeFull Support. Requires an x-speakeasy-type-override: any annotation. Used as an escape-hatch.
OpenAPI readOnly: trueFull Support.
OpenAPI writeOnly: trueFull Support.
Example GenerationFull Support. Propagates example and examples into generated terraform resource examples: but otherwise uses a type-appropriate value.
Massaging of divergent API and Schema Types⚠️Partial Support. We support a lot of different mechanisms to configure reconciliation of divergent terraform types, and merging together of multiple API requests into a singular resource schema even when there's type-divergence across APIs. Reach out to us on Slack support to see how we do this, or have a look through these Advanced Examples that describe some common scenarios.

What are Speakeasy's Limitations

While Speakeasy covers a wide range of OpenAPI features, we're constantly working to support even more. The following are current limitations with our terraform provider generation product:

Limitations
OpenAPI SemanticsSpeakeasy Support / Comment
Operation-specific Authentication⚠️Speakeasy supports all OpenAPI global authentication mechanisms: however there is currently no support for overriding this on some specific operations (without advanced techniques like monkey patching). Ensure that all API operations are authenticated through global security configuration.
label / matrix Path Parameter Serialization⚠️No support for label or matrix path parameter styles. Might require remapping through x-speakeasy-match when path parameter names for Read/Update/Delete do not match Create attribute names.
XML Request Body Serialization⚠️Full Support for all JSON data types, multipart encoding, binary, form data, file uploads, urlencoded form methods, plain text, raw byte entries. No support for XML or other methods. All request attributes will be available as resource/data source attributes in a terraform-native form.
XML Response Body Deserialization⚠️Full Support for all JSON data types, Plain Text, Raw Bytes. No support for other media types. All response attributes will be made available in terraform state
Circular References⚠️Partial Support through x-speakeasy-type-override: any to enable an attribute to be set with jsonencode(...arbitrary data...). However unfortunately the terraform type schema doesn't support circular dependencies
Lists of Lists of Primitives⚠️Partial Support through x-speakeasy-type-override: any to enable an attribute to be set with jsonencode(...arbitrary data...). Unfortunately the terraform ListAttribute only allows for primitive ItemTypes within. To support Lists of Lists of attributes, we need to use an escape hatch (like x-speakeasy-ignore, x-speakeasy-type-override: any
Runtime validations (e.g. pattern)⚠️Partial Support. Contact support if you need more.

Note: Circular References

The Terraform Type Schema must be a Direct Acyclic Graph. OpenAPI's use of JSON Schema allows for Circular references to be defined. When a Circular Reference is detected, we will currently output an error, and instructions for a few workarounds.

To solve this, apply x-speakeasy-type-override: any to the attribute that is causing the circular reference. This will cause the attribute to be inferred as a String that satisfies JSON. A terraform user can then user the jsonencode function to pass arbitrary data into the attribute.

Terraform Framework Types from JSON Schema Types

Attribute Types are always inferred, with one exception (see below)

Exceptions
JSON Schema TypeTerraform Framework TypeExceptions
type: stringschema.StringAttribute
type: numberschema.NumberAttribute
type: integerschema.Int64Attribute
type: booleanschema.BoolAttribute
type: arrayschema.ListAttribute or Schema.ListNestedAttributeA ListAttribute is used when the array has primitive children.
type: objectschema.SingleNestedAttribute
type: nullElement Ignored
additionalPropertiesschema.MapNestedAttributeThe item type of the MapNestedAttribute is dependent on the subschema of the additionalProperties value
JSON Schema SubSchema TypeHandling
oneOfA schema.SingleNestedAttribute is created with one key per oneOf child. If a discriminator is defined, the discriminator is used to determine the name of subkey. A plan validator is defined to validate only one sub attribute is in use at one time.
anyOfConsidered the same as oneOf. We do not strictly handle this subschema type due to never seeing it being used in production correctly.
allOfAll the subschemas are "merged" together. If the child schemas are objects, a derived object is created with all the properties of the children within.

1. Prerequisites

To get started with the Speakeasy Terraform creation, you will need:

Spec FormatSupported
OpenAPI 3.0
OpenAPI 3.1
JSON Schema
Postman Collection🔜
Success Icon

TIP

If you are using an unsupported spec format, some tools can help convert you to a supported format:

2. Add Annotations

Annotate objects representing Terraform entities with x-speakeasy-entity. This determines their inclusion in the Terraform provider.


paths:
/pet:
post:
...
x-speakeasy-entity-operation: Pet#create
...
Pet:
x-speakeasy-entity: Pet
...

Terraform Usage:


resource "petstore_pet" "myPet" {
...
}

Speakeasy infers Terraform types from your JSON schema, focusing on the semantics of CREATE and UPDATE requests and responses. You should not need to define any specific Terraform types in your OpenAPI spec.

  1. Required vs Optional: A property is marked as Required: true if it's required in the CREATE request body; otherwise, it's Optional: true.
  2. Computed Properties: Properties that appear in a response body but are absent from the CREATE request are marked as Computed: true. This indicates that Terraform will compute their values
  3. ForceNew Property: If a property exists in the CREATE request but is not present in the UPDATE request, it's labeled ForceNew.
  4. Enum Validation: When an attribute is defined as an enum, Speakeasy configures a Validator for runtime type checks. This ensures that all request properties precisely match one of the enumerated values.
  5. READ, UPDATE, DELETE Dependencies: Every parameter essential for READ, UPDATE, or DELETE operations must either be part of the CREATE API response body or be consistently required in the CREATE API request. This ensures that all necessary parameters are available for these operations.
Success Icon

TIP

Use additional x-speakeasy annotations to customize your provider as needed.

4. Enhance Generated Documentation

Speakeasy aids in auto-generating documentation using HashiCorp's terraform-plugin-docs. To make the most of this, we recommend:

  1. Including Descriptions: Ensure that your OpenAPI spec contains detailed descriptions of resources, attributes, and operations. Clear and concise descriptions help users understand the purpose and usage of each component.
  2. Providing Examples: Use examples in your OpenAPI spec to illustrate how resources and attributes should be configured. Speakeasy leverages these examples to generate usage snippets that users can refer to when starting with your provider.

The Swagger Pet Store would generate a usage snippet for the pet resource like the following:

"petstore_pet"

id = 10
name = "doggie"
photo_urls = [
"...",
]
}.

5. Generate Terraform

  • Run the Speakeasy generate command:

speakeasy generate sdk --lang terraform -o . -s [openAPI spec]

Frequently Asked Questions

Do Generated Terraform Providers Support the Ability to Import Resources?

Yes, generated Terraform providers do support the ability to import resources. However, there are specific prerequisites and considerations to keep in mind:

Prerequisites

  1. API Specification: It is essential to have an annotated and type-complete API operation defined for READing each resource in the OpenAPI specification. This must be tagged with x-speakeasy-entity-operation: MyEntity#read.

  2. Single Resource ID: Currently, we only support generating import logic for resources that can be READ using a single ID-like attribute. An ID-like attribute refers to any required attribute in the READ API request.

  3. Comprehensive READ Operation: If any attribute of a resource is not defined in the READ API, Terraform will set that attribute to `null’ during the import process.

Handling Composite Keys

In scenarios with multiple ID fields, direct import is not natively supported. A workaround is to serialize composite keys into a single ID field and override the implementation by parsing the Import ID request.

Composite Key Format: "<sourceId>,<workspaceID>"