Autoupdate
Auto Update is a tool for package maintainers. It automatically checks for new versions of an app and updates the manifest accordingly. It helps to eliminate much of the tedium of updating manifests, as well as reducing the risk of human error while doing so.
Here you will find an in-depth explanation of how the autoupdate
part of an app manifest works.
checkver.ps1
to query and autoupdate
Using Use checkver.ps1
to query the current version of either a specific app or all apps of a bucket. If an updated version existed, you can autoupdate manifest by checkver.ps1
too.
Querying current version
Open a PowerShell/CMD console, then cd
into Scoop's root directory (apps\scoop\current
) or the buckets repository directory and run the following commands:
To query the current version of a specific app in the bucket, run:
.\bin\checkver.ps1 <app> # or .\bin\checkver.ps1 -App <app>
To query the current version of all apps in the bucket, run:
.\bin\checkver.ps1 * # or .\bin\checkver.ps1 -App *
Updating manifest
In the output of checkver.ps1
, you can see if an outdated app has autoupdate available. If so, you can run the following command to automatically update the respective app's manifest (use *
to update all apps)
.\bin\checkver.ps1 <app> -u
# or .\bin\checkver.ps1 -App <app> -Update
To query apps that not in bucket
directory (e.g. .
, .\TODO
, etc.), specify the directory as the second argument to checkver.ps1
.\bin\checkver.ps1 <app> <dir> -u
# or .\bin\checkver.ps1 -App <app> -Dir <dir> -Update
It is recommended to verify that the updated manifest still works by installing the app with the following command
scoop install bucket\<app>.json
checkver.ps1
Parameters of -App
(-a APP
)- Manifest name to search.
- Placeholders (
*
) are supported.
-Dir
(-d DIR
)- Where to search for manifest(s).
-Update
(-u
)- Update given manifest.
-ForceUpdate
(-f
)- Check manifest(s) and update, even if there is no new version.
-SkipUpdated
(-s
)- Check given manifest(s), and list only outdated manifest(s).
-Version
(-v VER
)- Check given manifest(s) using a given version
VER
. - Usually used with
-u
to update to a certain version.
- Check given manifest(s) using a given version
checkver
to a manifest
Adding checkver
Using RegEx in Simplest solution is to use an RegEx and it will match it to the source of homepage
. Example: go
{
"homepage": "<https://golang.org>",
"checkver": "Build version go([\\d\\.]+)\\."
}
If you're not familiar with RegEx or want to test if your RegEx matches on the right text you can use an online tool (RegEx101 or RegExr).
Use another url if the homepage
doesn't contain the version. Example: 7zip
{
"homepage": "https://www.7-zip.org/",
"checkver": {
"url": "https://www.7-zip.org/download.html",
"regex": "Download 7-Zip ([\\d.]+)"
}
}
checkver
Using JSONPath in Use a JSON endpoint with JSONPath expressions to retrieve the version. Either dot-notation or bracket-notation can be used. Example: mro
{
"checkver": {
"url": "https://mran.microsoft.com/assets/configurations/app.config.json",
"jp": "$.latestMicrosoftRVersion"
}
}
If you're not familiar with JSONPath or want to test if your JSONPath matches on the right text you can use an online tool (JSONPath Expression Tester).
There could be a JSONPath query in checkver.jsonpath
, and so does RegEx (sample reference). Example: nuget
{
"checkver": {
"url": "https://dist.nuget.org/index.json",
"jp": "$..versions[?(@.displayName == 'nuget.exe - recommended latest')].version"
}
}
checkver
Using RegEx with JSONPath in If checkver.regex
and checkver.jsonpath
are all assigned, scoop use checkver.jsonpath
to extract a string which checkver.regex
is matched to to find the version. Example: nwjs
{
"checkver": {
"url": "https://nwjs.io/versions.json",
"jsonpath": "$.stable",
"regex": "v([\\d.]+)"
}
}
Special Cases
Use the latest app release on Github by setting checkver
to github
and the homepage
to the repository URL. This will try to match the tag with \/releases\/tag\/(?:v|V)?([\d.]+)
. The app author has to use Github's release feature for this to work. Pre-releases will be ignored! Example: nvm
{
"homepage": "https://github.com/coreybutler/nvm-windows",
"checkver": "github"
}
Or use different urls for the homepage and repository. Example: cmder
{
"homepage": "http://cmder.net",
"checkver": {
"github": "https://github.com/cmderdev/cmder"
}
}
Use checkver.reverse: true
to let checkver.regex
match the last occurrence found (default is to match the first occurrence). Example: x264
{
"checkver": {
"url": "https://download.videolan.org/pub/videolan/x264/binaries/win64/",
"re": "x264-r(?<version>[\\d]+)-(?<commit>[a-fA-F0-9]{7}).exe",
"reverse": true
}
}
Use capture groups in checkver.regex
will make captured variables that could be used in checkver.replace
for complex versions or in autoupdate
property.
This example will provide $matchVersion
and $matchShort
as variables (used in autoupdate
). Example: git
{
"checkver": {
"url": "https://github.com/git-for-windows/git/releases/latest",
"re": "v(?<version>[\\d\\w.]+)/PortableGit-(?<short>[\\d.]+).*\\.exe"
}
}
This exmple will provide ${1}, ${2}, ${3}
(used in checkver.replace
) and $matchSha
(used in autoupdate
) as variables. Example: pshazz
{
"checkver": {
"url": "https://github.com/lukesampson/pshazz/commits/master.atom",
"re": "(\\d+)-(\\d+)-(\\d+)[\\S\\s]*?(?<sha>[0-9a-f]{40})",
"replace": "0.${1}.${2}.${3}"
}
}
checkver
Properties of checkver
: "regex". RegEx for finding the version on thehomepage
github
: "uri". URL to the app's Github repositoryurl
: "uri". Page where the version can be found- Supports version variables
regex|re
: "regex". RegEx for finding the versionjsonpath|jp
: "jsonpath". JSONPath expression for finding the versionxpath
: "string". XPath expression for finding the versionreverse
: "boolean". If or not match the last occurrence foundreplace
: "string". Replace the matched value with a calculated value- Supports captured variables
useragent
: "string". User-Agent that used to get webpage content (only used in fiddler)- Supports version variables
autoupdate
to a manifest
Adding For the autoupdate feature to work it needs a checkver
property
to find the latest version number.
{
"autoupdate": {
"note": "Thanks for using autoupdate, please test your updates!",
"architecture": {
"64bit": {
"url": "https://example.org/dl/example-v$version-x64.msi"
},
"32bit": {
"url": "https://example.org/dl/example-v$version-x86.msi"
}
}
}
}
Some example manifests using the autoupdate
feature:
{
"autoupdate": {
"architecture": {
"64bit": {
"url": "https://nodejs.org/dist/v$version/node-v$version-win-x64.7z",
"extract_dir": "node-v$version-win-x64"
},
"32bit": {
"url": "https://nodejs.org/dist/v$version/node-v$version-win-x86.7z",
"extract_dir": "node-v$version-win-x86"
}
},
"hash": {
"url": "$baseurl/SHASUMS256.txt.asc"
}
}
}
{
"autoupdate": {
"architecture": {
"64bit": {
"url": "https://windows.php.net/downloads/releases/php-$version-Win32-VC15-x64.zip"
},
"32bit": {
"url": "https://windows.php.net/downloads/releases/php-$version-Win32-VC15-x86.zip"
}
},
"hash": {
"url": "$baseurl/sha256sum.txt"
}
}
}
{
"autoupdate": {
"url": "https://nginx.org/download/nginx-$version.zip",
"extract_dir": "nginx-$version"
}
}
{
"autoupdate": {
"architecture": {
"64bit": {
"url": "https://www.imagemagick.org/download/binaries/ImageMagick-$version-portable-Q16-x64.zip"
},
"32bit": {
"url": "https://www.imagemagick.org/download/binaries/ImageMagick-$version-portable-Q16-x86.zip"
}
},
"hash": {
"mode": "rdf",
"url": "https://www.imagemagick.org/download/binaries/digest.rdf"
}
}
}
Some examples using the autoupdate
feature with captured variables or version variables:
{
"checkver": {
"re": "/(?<type>early_access|GA)/(?<path>jdk(?<major>[\\d.]+)(?:.*)?/(?<build>[\\d]+)(?:/GPL|/binaries)?)/(?<file>openjdk-(?<version>[\\d.]+)(?<ea>-ea)?(?:\\+[\\d]+)?_windows-x64_bin.(zip|tar.gz))",
"replace": "${version}-${build}${ea}"
},
"autoupdate": {
"architecture": {
"64bit": {
"url": "https://download.java.net/java/$matchType/$matchPath/$matchFile"
}
},
"hash": {
"url": "$url.sha256"
},
"extract_dir": "jdk-$matchVersion"
}
}
{
"autoupdate": {
"architecture": {
"64bit": {
"url": "https://dev.mysql.com/get/Downloads/MySQL-$majorVersion.$minorVersion/mysql-$version-winx64.zip",
"extract_dir": "mysql-$version-winx64",
"hash": {
"url": "https://dev.mysql.com/downloads/mysql/",
"find": "md5\">([A-Fa-f\\d]{32})"
}
}
}
}
}
autoupdate
Properties of All the properties except autoupdate.note
can be set globally for all architectures or for each architecture separately (under architecture.64bit
or architecture.32bit
). Global properties can be used to update each architectural properties, i.e., if only setted globally, autoupdate.url
is used to update either architecture.64bit.url
or architecture.32bit.url
.
url
: "uri". An URL template for generating the new url- scoop will rename files by appending
#/dl.7z
or#/pngcrush.exe
to the URL (useful for extracting installers or renaming executables version string) - Supports captured variables
- Supports version variables
- scoop will rename files by appending
hash
: "object". Set this property for obtaining hash values without download the actual filesextract_dir
: "string". Option to updateextract_dir
- Supports captured variables
- Supports version variables
note
: "string". Optional message to be displayed when the autoupdate command is run
hash
to autoupdate
Adding There are several ways to obtain the hash of the new file. If the app provider publishes hash values it is possible to extract these from their website or hashfile. If nothing is defined or something goes wrong while getting the hash values the target files will be downloaded and hashed locally.
Hash value can be directly extracted by the following method (autoupdate.hash.mode
):
- Using RegEx for plain text file or webpage (
extract
, predefinedfosshub
,sourceforge
) - Using JSONPath for JSON file (
json
) - Using XPath for XML file (
xpath
) - Using Digest for RDF file (
rdf
) - Using download header or
.meta4
for Metalink (metalink
)
hash.url
Specifying URL in url
in hash
property accepts URL with captured variables, version variables or URL variables.
- Use captured variables. Example: qemu
{
"checkver": {
"re": "<strong>(?<year>\\d{4})-(?<month>\\d{2})-(?<day>\\d{2})</strong>: New QEMU installers \\((?<version>[\\d.a-z\\-]+)\\)"
},
"autoupdate": {
"architecture": {
"64bit": {
"url": "https://qemu.weilnetz.de/w64/qemu-w64-setup-$matchYear$matchMonth$matchDay.exe#/dl.7z",
"hash": {
"url": "https://qemu.weilnetz.de/w64/qemu-w64-setup-$matchYear$matchMonth$matchDay.sha512"
}
},
"32bit": {
"url": "https://qemu.weilnetz.de/w32/qemu-w32-setup-$matchYear$matchMonth$matchDay.exe#/dl.7z",
"hash": {
"url": "https://qemu.weilnetz.de/w32/qemu-w32-setup-$matchYear$matchMonth$matchDay.sha512"
}
}
}
}
}
- Use version variables. Example: julia
{
"hash": {
"url": "https://julialang-s3.julialang.org/bin/checksums/julia-$version.sha256"
}
}
- Use URL variables and append suffix to it. Example: apache
{
"hash": {
"url": "$url_fossies.sha256"
}
}
Getting hash from plain text file or webpage
If the app provider publishes hash value in a plain text file or on some webpage, checkver.ps1
could extract it by using proper RegEx.
autoupdate.hash
Using Built-in RegEx in There are some default RegEx that built in scoop, i.e., ^([a-fA-F0-9]+)$
and ([a-fA-F0-9]{32,128})[\x20\t]+.*$basename(?:[\x20\t]+\d+)?
.
# ^([a-fA-F0-9]+)$
abcdef0123456789abcdef0123456789abcdef01
# ([a-fA-F0-9]{32,128})[\x20\t]+.*`$basename(?:[\x20\t]+\d+)?
abcdef0123456789abcdef0123456789abcdef01 *example.zip
See above for examples.
autoupdate.hash
Using customized RegEx in If built-in RegEx is not suitable, customized RegEx could be used to extract hash value. Hash variables ($md5
, $sha1
, $sha256
, and etc.) could be used to simplify the expression.
{
"hash": {
"url": "$baseurl/hashes.txt",
"find": "SHA256\\($basename\\)=\\s+([a-fA-F\\d]{64})"
}
}
{
"hash": {
"url": "https://www.python.org/downloads/release/python-$cleanVersion/",
"find": "$basename[\\S\\s]+?$md5"
}
}
Special cases for FossHub and SourceForge
There are two predefined special cases: FossHub and SourceForge.
- FossHub
- URL pattern:
^(?:.*fosshub.com\/).*(?:\/|\?dwl=)(?<filename>.*)$
https://www.fosshub.com/Calibre.html?dwl=calibre-64bit-3.44.0.msi
https://www.fosshub.com/Calibre.html/calibre-64bit-3.44.0.msi
- RegEx:
$basename.*?"sha256":"([a-fA-F0-9]{64})"
- URL pattern:
- SourceForge
- URL pattern:
(?:downloads\.)?sourceforge.net\/projects?\/(?<project>[^\/]+)\/(?:files\/)?(?<file>.*)
https://downloads.sourceforge.net/project/nsis/NSIS%203/3.04/nsis-3.04.zip
https://sourceforge.net/projects/nsis/files/NSIS%203/3.04/nsis-3.04.zip
- RegEx:
"$basename":.*?"sha1":\s"([a-fA-F0-9]{40})"
- URL pattern:
For autoupdate.url
s that match above patterns, hash mode is setted to fosshub
or sourceforge
automatically and hash
property is not needed.
{
"autoupdate": {
"url": "https://www.fosshub.com/Calibre.html/calibre-portable-installer-$version.exe"
}
}
{
"autoupdate": {
"url": "https://downloads.sourceforge.net/project/nsis/NSIS%20$majorVersion/$version/nsis-$version.zip",
"extract_dir": "nsis-$version"
}
}
Getting hash from JSON file
For JSON file, use a JSON endpoint with JSONPath expressions to retrieve the hash. Either dot-notation or bracket-notation can be used. Example: openssl
{
"hash": {
"mode": "json",
"jp": "$.files.['$basename'].sha512",
"url": "$baseurl/win32_openssl_hashes.json"
}
}
There could be a JSONPath query in autoupdate.hash.jsonpath
, and so does RegEx (sample reference). Example: mro
{
"hash": {
"mode": "json",
"jsonpath": "$..versions[?(@.downloadText == '$basename')].sha256",
"url": "https://mranapi.azurewebsites.net/api/download"
}
}
Getting hash from XML file
Use XPath to retrieve the hash from a XML file. Example: googlechrome
{
"hash": {
"url": "https://lab.skk.moe/chrome/api/chrome.min.xml",
"xpath": "/chromechecker/stable64[version=\"$version\"]/sha256"
}
}
Getting hash from RDF file
Resource Description Framework (RDF) is a framework for representing information in the Web. Hash value could be extracted from RDF file. Example: imagemagick
{
"hash": {
"mode": "rdf",
"url": "https://www.imagemagick.org/download/binaries/digest.rdf"
}
}
Getting hash from from Metalink
Metalink is an Internet standard that harnesses the speed and power of peer to peer networking and traditional downloads with a single click. For download URL that supported Metalink, hash value could be retrieved from download URL's header or a .meta4
file. Example: libreoffice-fresh
{
"hash": {
"mode": "metalink"
}
}
autoupdate.hash
Properties of All the properties can be set globally for all architectures or for each architecture separately
mode
: "String | Enum".extract
: Extract from a plain text file or webpage by RegEx (Default, can be omitted)json
: Extract from a JSON file by JSONPathxpath
: Extract from a XML file by XPathrdf
: Extract from a RDF filemetalink
: Extract from Metalink's header or.meta4
filefosshub
: Automatic. Predefined for FossHubsourceforge
: Automatic. Predefined for SourceForgedownload
: Downloads the app file and hash it locally (Fallback)
url
: "Uri". URL template for downloading RDF/JSON files or extracting hashes- Supports captured variables
- Supports version variables
- Supports URL variables
regex|find
: "Regex". RegEx expression to extract the hash- Defaults:
^([a-fA-F0-9]+)$
and([a-fA-F0-9]{32,128})[\x20\t]+.*$basename(?:[\x20\t]+\d+)?
- Supports captured variables
- Supports version Variables
- Supports URL variables
- Supports hash variables
- Defaults:
jsonpath|jp
: "JSONPath". JSONPath expression to extract the hash- Supports captured variables
- Supports version Variables
- Supports URL variables
xpath
: "String". XPath expression to extract the hash- Supports captured variables
- Supports version Variables
- Supports URL variables
type
: "String | Enum". Deprecated. Hash type is determined automatically
Internal substitutable variables
Captured variables
- Used in
checkver.replace
${1}
,${2}
,${3}
...: Unnamed groups${name1}
,${Name2}
,${NAME3}
...: Named groups(?<name1>...)
,(?<Name2>...)
,(?<NAME3>...)
...
- Used in
autoupdate
$match1
,$match2
,$match3
...: Unnamed groups$matchName1
,$matchName2
,$matchName3
...: Named groups(?<name1>...)
,(?<Name2>...)
,(?<NAME3>...)
...- Notice the only uppercase character in variable name
Version variables
$version
:3.7.1
$underscoreVersion
:3_7_1
$dashVersion
:3-7-1
$cleanVersion
:371
- The
$version
(e.g.3.7.1.2
) is splitted on each.
and is assigned to:$majorVersion
:3
$minorVersion
:7
$patchVersion
:1
$buildVersion
:2
$matchHead
: Returns first two or three digits seperated by a dot (e.g.3.7.1-rc.1
=3.7.1
,3.7.1.2-rc.1
=3.7.1
or3.7-rc.1
=3.7
)$matchTail
: Returns the rest of$matchHead
(e.g.3.7.1-rc.1
=-rc.1
,3.7.1.2-rc.1
=.2-rc.1
or3.7-rc.1
=-rc.1
)$preReleaseVersion
: Everything after the last-
(e.g.3.7.1-rc.1
would result inrc.1
)- Each capturing group in the
checkver
property adds a$matchX
variable (named groups are allowed). Matchingv3.7.1/3.7
withv(?<version>[\d.]+)\/(?<short>[\d.]+)
would result in:$match1
or$matchVersion
:3.7.1
$match2
or$matchShort
:3.7
URL variables
$url
: autoupdate URL without fragments (#/dl.7z
) [e.g.http://example.com/path/file.exe
]$baseurl
: autoupdate URL without filename and fragments (#/dl.7z
) [e.g.http://example.com/path
]$basename
: filename from autoupdate URL (ignores fragments#/dl.7z
)
Hash variables
$md5
:([a-fA-F0-9]{32})
MD5 hash type$sha1
:([a-fA-F0-9]{40})
SHA-1 hash type$sha256
:([a-fA-F0-9]{64})
SHA-256 hash type$sha512
:([a-fA-F0-9]{128})
SHA-512 hash type$checksum
:([a-fA-F0-9]{32,128})
MD5, SHA-1, SHA-256 or SHA-512 hash type$base64
:([a-zA-Z0-9+\/=]{24,88})
BASE64 encoded checksum (can be MD5, SHA-1, SHA-256 or SHA-512)
Limitations
There are some complex manifests which reach the limits of the current autoupdate implementation, mainly because autoupdate
only update url
, hash
and extract_dir
. (The list of affected manifests is incomplete)
- The binaries specified in the
bin
orshortcuts
change with the version number. Example: gimp - There are multiple
url
s needed to be updated. Example: coreutils
Testing and running autoupdate
If you want to confirm an autoupdate works, e.g. after adding it to an existing manifest or creating a new one, change the version
field to a lower or different version and then run checkver.ps1
or use the -f
parameter.
cd <bucket repository>
scoop config debug $true
.\bin\checkver.ps1 <app> -u
Check if the url
, hash
and extract_dir
properties have the correct values. Try to install/uninstall the app and submit your changes.
Manifests in some known buckets are autoupdated by ScoopInstaller/Excavator, so if you want some apps being autoupdated, migrate them to one of these buckets or run an instance of the excavator yourself.
main
: Update per hourextras
: Update per hourversions
: Update per dayjava
: Update per dayphp
: Update per daygames
: Update per day
Example Workflow with scoop status/update
scoop status
compares your installed version against the current copy of scoop and bucket repositories on your machine. If these are not updated it will output wrong version information. e.g.:
- installed version: 2.1.2
- local scoop version: 2.1.3
- online repo version: 2.1.4
scoop status
will show version 2.1.3
Running scoop update
before scoop status
is recommended (which is enforced every 3 hours), then it will show currect version 2.1.4.
scoop update
just git pull
s scoop core repo to ~\scoop\apps\scoop\current
and every configured bucket to ~\scoop\buckets\<name>
(incl. default main bucket)
bin\checkver.ps1
is only for maintenance and updating the manifests so they can be committed to the repo.
Example Workflow:
cd <bucket repository>
.\bin\checkver * -u # updates all manifest in the repo
git commit -m "Updated apps"
git push
scoop update
scoop status
scoop update <app>