Upgrade guide¶
Datasette 0.X -> 1.0¶
This section reviews breaking changes Datasette 1.0 has when upgrading from a 0.XX version. For new features that 1.0 offers, see the Changelog.
New URL for SQL queries¶
Prior to 1.0a14 the URL for executing a SQL query looked like this:
/databasename?sql=select+1
# Or for JSON:
/databasename.json?sql=select+1
This endpoint served two purposes: without a ?sql= it would list the tables in the database, but with that option it would return results of a query instead.
The URL for executing a SQL query now looks like this:
/databasename/-/query?sql=select+1
# Or for JSON:
/databasename/-/query.json?sql=select+1
This isn't a breaking change. API calls to the older /databasename?sql=... endpoint will redirect to the new databasename/-/query?sql=... endpoint. Upgrading to the new URL is recommended to avoid the overhead of the additional redirect.
Metadata changes¶
Metadata was completely revamped for Datasette 1.0. There are a number of related breaking changes, from the metadata.yaml file to Python APIs, that you'll need to consider when upgrading.
metadata.yaml split into datasette.yaml¶
Before Datasette 1.0, the metadata.yaml file became a kitchen sink if a mix of metadata, configuration, and settings. Now metadata.yaml is strictly for metadata (ex title and descriptions of database and tables, licensing info, etc). Other settings have been moved to a datasette.yml configuration file, described in Configuration.
To start Datasette with both metadata and configuration files, run it like this:
datasette --metadata metadata.yaml --config datasette.yaml
# Or the shortened version:
datasette -m metadata.yml -c datasette.yml
Upgrading an existing metadata.yaml file¶
The datasette-upgrade plugin can be used to split a Datasette 0.x.x metadata.yaml (or .json) file into separate metadata.yaml and datasette.yaml files. First, install the plugin:
datasette install datasette-upgrade
Then run it like this to produce the two new files:
datasette upgrade metadata-to-config metadata.json -m metadata.yml -c datasette.yml
Metadata "fallback" has been removed¶
Certain keys in metadata like license used to "fallback" up the chain of ownership.
For example, if you set an MIT to a database and a table within that database did not have a specified license, then that table would inherit an MIT license.
This behavior has been removed in Datasette 1.0. Now license fields must be placed on all items, including individual databases and tables.
The get_metadata() plugin hook has been removed¶
In Datasette 0.x plugins could implement a get_metadata() plugin hook to customize how metadata was retrieved for different instances, databases and tables.
This hook could be inefficient, since some pages might load metadata for many different items (to list a large number of tables, for example) which could result in a large number of calls to potentially expensive plugin hook implementations.
As of Datasette 1.0a14 (2024-08-05), the get_metadata() hook has been deprecated:
# ❌ DEPRECATED in Datasette 1.0
@hookimpl
def get_metadata(datasette, key, database, table):
pass
Instead, plugins are encouraged to interact directly with Datasette's in-memory metadata tables in SQLite using the following methods on the Datasette class:
A plugin that stores or calculates its own metadata can implement the startup(datasette) hook to populate those items on startup, and then call those methods while it is running to persist any new metadata changes.
The /metadata.json endpoint has been removed¶
As of Datasette 1.0a14, the root level /metadata.json endpoint has been removed. Metadata for tables will become available through currently in-development extras in a future alpha.
The metadata() method on the Datasette class has been removed¶
As of Datasette 1.0a14, the .metadata() method on the Datasette Python API has been removed.
Instead, one should use the following methods on a Datasette class:
Datasette 1.0a20 plugin upgrade guide¶
Datasette 1.0a20 makes some breaking changes to Datasette's permission system. Plugins need to be updated if they use any of the following:
The
register_permissions()plugin hook - this should be replaced withregister_actionsThe
permission_allowed()plugin hook - this should be upgraded to usepermission_resources_sql().The
datasette.permission_allowed()internal method - this should be replaced withdatasette.allowed()Logic that grants access to the
"root"actor can be removed.
Permissions are now actions¶
The register_permissions() hook shoud be replaced with register_actions().
Old code:
@hookimpl
def register_permissions(datasette):
return [
Permission(
name="explain-sql",
abbr=None,
description="Can explain SQL queries",
takes_database=True,
takes_resource=False,
default=False,
),
Permission(
name="annotate-rows",
abbr=None,
description="Can annotate rows",
takes_database=True,
takes_resource=True,
default=False,
),
Permission(
name="view-debug-info",
abbr=None,
description="Can view debug information",
takes_database=False,
takes_resource=False,
default=False,
),
]
The new Action does not have a default= parameter.
Here's the equivalent new code:
from datasette import hookimpl
from datasette.permissions import Action
from datasette.resources import DatabaseResource, TableResource
@hookimpl
def register_actions(datasette):
return [
Action(
name="explain-sql",
description="Explain SQL queries",
resource_class=DatabaseResource,
),
Action(
name="annotate-rows",
description="Annotate rows",
resource_class=TableResource,
),
Action(
name="view-debug-info",
description="View debug information",
),
]
The abbr= is now optional and defaults to None.
For actions that apply to specific resources (like databases or tables), specify the resource_class instead of takes_parent and takes_child. Note that view-debug-info does not specify a resource_class because it applies globally.
permission_allowed() hook is replaced by permission_resources_sql()¶
The following old code:
@hookimpl
def permission_allowed(action):
if action == "permissions-debug":
return True
Can be replaced by:
from datasette.permissions import PermissionSQL
@hookimpl
def permission_resources_sql(action):
return PermissionSQL.allow(reason="datasette-allow-permissions-debug")
A .deny(reason="") class method is also available.
For more complex permission checks consult the documentation for that plugin hook: https://docs.datasette.io/en/latest/plugin_hooks.html#permission-resources-sql-datasette-actor-action
Using datasette.allowed() to check permissions instead of datasette.permission_allowed()¶
The internal method datasette.permission_allowed() has been replaced by datasette.allowed().
The old method looked like this:
can_debug = await datasette.permission_allowed(
request.actor,
"view-debug-info",
)
can_explain_sql = await datasette.permission_allowed(
request.actor,
"explain-sql",
resource="database_name",
)
can_annotate_rows = await datasette.permission_allowed(
request.actor,
"annotate-rows",
resource=(database_name, table_name),
)
Note the confusing design here where resource could be either a string or a tuple depending on the permission being checked.
The new keyword-only design makes this a lot more clear:
from datasette.resources import DatabaseResource, TableResource
can_debug = await datasette.allowed(
actor=request.actor,
action="view-debug-info",
)
can_explain_sql = await datasette.allowed(
actor=request.actor,
action="explain-sql",
resource=DatabaseResource(database_name),
)
can_annotate_rows = await datasette.allowed(
actor=request.actor,
action="annotate-rows",
resource=TableResource(database_name, table_name),
)
Root user checks are no longer necessary¶
Some plugins would introduce their own custom permission and then ensure the "root" actor had access to it using a pattern like this:
@hookimpl
def register_permissions(datasette):
return [
Permission(
name="upload-dbs",
abbr=None,
description="Upload SQLite database files",
takes_database=False,
takes_resource=False,
default=False,
)
]
@hookimpl
def permission_allowed(actor, action):
if action == "upload-dbs" and actor and actor.get("id") == "root":
return True
This is no longer necessary in Datasette 1.0a20 - the "root" actor automatically has all permissions when Datasette is started with the datasette --root option.
The permission_allowed() hook in this example can be entirely removed.
Root-enabled instances during testing¶
When writing tests that exercise root-only functionality, make sure to set datasette.root_enabled = True on the Datasette instance. Root permissions are only granted automatically when Datasette is started with datasette --root or when the flag is enabled directly in tests.
Target the new APIs exclusively¶
Datasette 1.0a20’s permission system is substantially different from previous releases. Attempting to keep plugin code compatible with both the old permission_allowed() and the new allowed() interfaces leads to brittle workarounds. Prefer to adopt the 1.0a20 APIs (register_actions, permission_resources_sql(), and datasette.allowed()) outright and drop legacy fallbacks.
Fixing async with httpx.AsyncClient(app=app)¶
Some older plugins may use the following pattern in their tests, which is no longer supported:
app = Datasette([], memory=True).app()
async with httpx.AsyncClient(app=app) as client:
response = await client.get("http://localhost/path")
The new pattern is to use ds.client like this:
ds = Datasette([], memory=True)
response = await ds.client.get("/path")
Migrating from metadata= to config=¶
Datasette 1.0 separates metadata (titles, descriptions, licenses) from configuration (settings, plugins, queries, permissions). Plugin tests and code need to be updated accordingly.
Update test constructors¶
Old code:
ds = Datasette(
memory=True,
metadata={
"databases": {
"_memory": {"queries": {"my_query": {"sql": "select 1", "title": "My Query"}}}
},
"plugins": {
"my-plugin": {"setting": "value"}
}
}
)
New code:
ds = Datasette(
memory=True,
config={
"databases": {
"_memory": {"queries": {"my_query": {"sql": "select 1", "title": "My Query"}}}
},
"plugins": {
"my-plugin": {"setting": "value"}
}
}
)
Update datasette.metadata() calls¶
The datasette.metadata() method has been removed. Use these methods instead:
Old code:
try:
title = datasette.metadata(database=database)["queries"][query_name]["title"]
except (KeyError, TypeError):
pass
New code:
try:
query_info = await datasette.get_canned_query(database, query_name, request.actor)
if query_info and "title" in query_info:
title = query_info["title"]
except (KeyError, TypeError):
pass
Update render functions to async¶
If your plugin's render function needs to call datasette.get_canned_query() or other async Datasette methods, it must be declared as async:
Old code:
def render_atom(datasette, request, sql, columns, rows, database, table, query_name, view_name, data):
# ...
if query_name:
title = datasette.metadata(database=database)["queries"][query_name]["title"]
New code:
async def render_atom(datasette, request, sql, columns, rows, database, table, query_name, view_name, data):
# ...
if query_name:
query_info = await datasette.get_canned_query(database, query_name, request.actor)
if query_info and "title" in query_info:
title = query_info["title"]
Update query URLs in tests¶
Datasette now redirects ?sql= parameters from database pages to the query view:
Old code:
response = await ds.client.get("/_memory.atom?sql=select+1")
New code:
response = await ds.client.get("/_memory/-/query.atom?sql=select+1")