Complete Guide for Seven Bridges API R Client

Quickstart

For more details about how to use the API client in R, please go for the second section for complete guide. This section, I am going to use a simple example for a quick start.

Create Auth object

Everything starts from an Auth object, so let’s set up the Auth object, it remembers your auth token and url, every action started from this object.

You have three different ways to setup the token.

Direct setup via Auth function, explicityly setup token and url.
Configuration file in your home folder “.sbg.auth.yml”, very easy to manage and implicitly loaded, everytime you start a new R session.
Tempoerary setup via option in R session

library(sevenbridges)
## direct setup
a <- Auth(token = "<fake_token>",
          url = "https://cgc-api.sbgenomics.com/v2/")
## or load default from config file (autoloaded into options)
library(sevenbridges)
a <- Auth(platform = "us", username = "tengfei")

== Auth ==
token : <fake_token>
url : https://cgc-api.sbgenomics.com/v2/

Infomation about a user

This call returns information about your account.

a$user()

== User ==
href : https://cgc-api.sbgenomics.com/v2/users/tengfei
username : tengfei
email : tengfei.yin@sbgenomics.com
first_name : Tengfei
last_name : Yin
affiliation : Seven Bridges Genomics
country : United States

To list user resrouces, This call returns information about the specified user. Note that currently you can view only your own user information, and so this call is equivalent to the call to Get my information.

a$user(username = "tengfei")

Rate limit

This call returns information about your current rate limit. This is the number of API calls you can make in one hour.

a$rate_limit()

== Rate Limit ==
limit : 1000
remaining : 993
reset : 1457980957

Show billing information

Billing information, every project is associated with a billing group

## check your billing info
a$billing()
a$invoice()

For more information, use breakdown = TRUE

a$billing(id = "your_billing_id", breakdown = TRUE)

Create a project

Create a new project called “api testing”, with the billing group id.

## get billing group id
bid <- a$billing()$id
## create new project
(p <- a$project_new(name = "api testing", bid, description = "Just a testing"))

== Project ==
id : tengfei/api-testing
name : api testing
description : Just a testing
billing_group_id : <fake_bid>
type : v2
-- Permission --

Import CWL app and run a task

Then we want to run a task, and you want to upload your json file that describe your tool, here we use an random number generator example saved in this package.

Note: alternatively you can directly describe your CWL tool in R with this package, please read another vignettes on “Describe CWL Tools/Workflows in R and Execution”

## Add an CWL file to your project
fl.runif <- system.file("docker", "sevenbridges/rabix/runif.json", package = "sevenbridges")

Let’s take a look at the content of this json file

cat(readLines(fl.runif), sep = '\n')

{
  "sbg:id": "runif",
  "id": "#runif",
  "inputs": [
    {
      "type": [
        "int"
      ],
      "label": "number",
      "description": "number of observations",
      "streamable": false,
      "default": 1,
      "id": "#number",
      "inputBinding": {
        "position": 0,
        "prefix": "--n",
        "separate": true,
        "sbg:cmdInclude": true
      }
    },
    {
      "type": ["null", "float"],
      "label": "min",
      "description": "lower limits of the distribution",
      "streamable": false,
      "default": 0,
      "id": "#min",
      "inputBinding": {
        "position": 0,
        "prefix": "--min",
        "separate": true,
        "sbg:cmdInclude": true
      }
    },
    {
      "type": ["null", "float"],
      "label": "max",
      "description": "upper limits of the distribution",
      "streamable": false,
      "default": 1,
      "id": "#max",
      "inputBinding": {
        "position": 0,
        "prefix": "--max",
        "separate": true,
        "sbg:cmdInclude": true
      }
    },
    {
      "type": ["null", "float"],
      "label": "seed",
      "description": "seed with set.seed",
      "streamable": false,
      "default": 1,
      "id": "#seed",
      "inputBinding": {
        "position": 0,
        "prefix": "--seed",
        "separate": true,
        "sbg:cmdInclude": true
      }
    }
  ],
  "outputs": [
    {
      "type": ["null", "File"],
      "label": "output",
      "description": "random number file",
      "streamable": false,
      "default": "",
      "id": "#random",
      "outputBinding": {
        "glob": "*.txt"
      }
    },
    {
      "type": ["null", "File"],
      "label": "report",
      "description": "",
      "streamable": false,
      "default": "",
      "id": "#report",
      "outputBinding": {
        "glob": "*.html"
      }
    }
  ],
  "requirements": [],
  "hints": [
    {
      "class": "DockerRequirement",
      "dockerPull": "tengfei/runif",
      "dockerLoad": "",
      "dockerFile": "",
      "dockerImageId": "",
      "dockerOutputDirectory": ""
    },
    {
      "class": "sbg:CPURequirement",
      "value": 1
    },
    {
      "class": "sbg:MemRequirement",
      "value": 2000
    }
  ],
  "label": "Random number generator",
  "class": "CommandLineTool",
  "baseCommand": [
    "runif.R"
  ],
  "arguments": [],
  "context": ""
}

Now let’s add this json file to your projct as a new app.

## name your app 
(aid <- p$app_add("runif", fl.runif)$id)

You get an id like this

"tengfei/api-testing/runif/0"

It’s composed of

project id : tengfei/api
app short name : runif
revision : 0

Alternatively, you can describe tools like this

fl <- system.file("docker", "sevenbridges/rabix/generator.R", package = "sevenbridges")
cat(readLines(fl), sep = '\n')

library(sevenbridges)
in.lst <- list(input(id = "number",
                     description = "number of observations",
                     type = "integer",
                     label = "number",
                     prefix = "--n",
                     default = 1,
                     required = TRUE, 
                     cmdInclude = TRUE),
               input(id = "min",
                     description = "lower limits of the distribution",
                     type = "float",
                     label = "min",
                     prefix = "--min",
                     default = 0),
               input(id = "max",
                     description = "upper limits of the distribution",
                     type = "float",
                     label = "max",
                     prefix = "--max",
                     default = 1),
               input(id = "seed",
                     description = "seed with set.seed",
                     type = "float",
                     label = "seed",
                     prefix = "--seed",
                     default = 1))


## the same method for outputs
out.lst <- list(output(id = "random",
                       type = "file",
                       label = "output", 
                       description = "random number file",
                       glob = "*.txt"),
                output(id = "report",
                       type = "file",
                       label = "report", 
                       glob = "*.html"))


rbx <- Tool(id = "runif",
            label = "Random number generator",
            hints = requirements(docker(pull = "tengfei/runif"), 
                                 cpu(1), mem(2000)),
            baseCommand = "runif.R",
            inputs = in.lst, ## or ins.df
            outputs = out.lst)


fl <- "inst/docker/sevenbridges/rabix/runif.json"
write(rbx$toJSON(pretty = TRUE), fl)

And add it like this

## rbx is the object returned by Tool function
(aid <- p$app_add("runif", rbx)$id)

Please read another tutorial about how to describe tools and flows in R.

Excute a new task

Now we draft a new task, you need to specify

name of the task
description
app id
inputs of your task: you need to know about the app you are running, in this case, the cwl app accept 4 parameters (number, min, max, seed).

I want to generate 1 random number (default) between 1 and 10.

(tsk <- p$task_add(name = "randome number 5", 
           description = "Generate some random number", 
           app = aid,
           inputs = list(min = 1, max = 10, number = 1, seed = 4)))

## confirm, show all task status is draft
p$task(status = "draft")

## or just list the task
tsk$update()

Now you have your draft task, you can delete your draft task or update it.

## not run
## tsk$delete()

looks like there is only a single number, oops, I want 100 numbers instead, so let me update my draft task

tsk$getInputs()
## missing number input, only update number
tsk$update(inputs = list(number = 500, seed = 1))
## double check 
tsk$getInputs()

Or we just want to run it in the cloud!

## Run your task
tsk$run()

To monitor the task, you can always call update on task object to check the status.

tsk$update()

Or more fun, you can monitor a running task with hook function, so trigger a function when that status is “completed”, “running” etc, please check the details in section about hook of task.

By default it just show message when the task is completed.

## Monitor your task (skip this part)
## tsk$monitor()

To about running task just all

## not run
## tsk$abort()

To download all files from a completed tasks

tsk$download("~/Downloads")

More fun to set task hook, so when it’s complete download the files

setTaskHook("completed", function(){
    tsk$download("~/Downloads")
})
tsk$monitor()

User-friendly API

This is what the package try to help, and provide a user-friendly interface that we suggest our users to use, so you don’t have to combine several api() calls and refer to the API documentation all the times to finish a simple task.

Authentification

Set up default token for different platforms

You can create a file called ‘.sbg.auth.yml’ in your home folder, and maintain multiple account for a list of platforms, including private or public ones.

us:
  url: https://api.sbgenomics.com/v2/
  user:
    tengfei:
      token: fake_token
    yintengfei:
      token: fake_token
cgc:
  url: https://cgc-api.sbgenomics.com/
  user:
    tengfei:
      token: fake_token
gcp:
  url: https://gcp-api.sbgenomics.com/v2/
  user:
    tengfei:
      token: fake_token

When you load sevenbridges package, it will first try to parse your token configuration file first into an options list.

## Create Auth object from config file
a <- Auth(username = "yintengfei", platform = "us")
## show all 
getToken()
## show all pre-set user token for platform 
getToken("cgc")
## show individual token for a user
getToken(platform = "cgc", username = "tengfei")

Note: when you edit your .sbg.auth.yml, you have to reload your package.

Create Auth object directly

First thing first, you need to construct an Auth object, everything begins with this object, it stores

The authentication token
The API URL
The platform (US platform, Cancer Genomics Cloud etc), this is optional, will translate into API url.

The logic is like this

If you didn’t pass url or token, we think you are loading from config file
if no platform or user provided, will use the first item in your token config file, this is not recommended, at least provide platform/username set.

library(sevenbridges)
## direct setup
a <- Auth(token = "1c0e6e202b544030870ccc147092c257",
          url = "https://cgc-api.sbgenomics.com/v2/")

By default it points to Cancer Genomics Cloud platform, unless you specify

API URL (more flexible)
or Platform (currently support ‘cgc’, ‘us’, ‘gcp’)

Note: when you construct the Auth object, make sure you input the correct platform or API url for your authentication. On Seven Bridges related platforms, you can always find it under your account setting and developer tab.

For the tutorial about how to get your authentication, please check

List All API calls

If we didn’t pass any parameters to api() from Auth, it will list all API calls, and anything parameter we provided will pass on to api() function, but you don’t need to input token and url again! The Auth object will know that information already.
And this call from Auth object will check the response too.

a$api()

Offset and Limit and Search

offset specify where it is started, and limit specify how many you want to show from there (max: 100). Because the item could be thousands of files and apps, so by default the offset and limit is set to 0 and 100 accordingly.

getOption("sevenbridges")$offset
getOption("sevenbridges")$limit

Please pay attention to this

Search by id is most accurate and fast for any Item like Project, App, Task, File.
Search by name will only search across current pulled content, so use complete = TRUE if you want to search across everything, this might be slow.

For example, to list all public apps, use visibility argument, but make sure you pass complete = TRUE to it, to show every single things. This arguments generally works for items like “App”, “Project”, “Task”, “File” etc

## first, search by id is fast
x <- a$app(visibility = "public", id = "djordje_klisic/public-apps-by-seven-bridges/sbg-ucsc-b37-bed-converter/0")

## show 100 items from public
x <-  a$app(visibility = "public")
length(x) ## 100


x <- a$app(visibility = "public", complete = TRUE)
length(x) ## 211 by March, 2016
## this return nothing, because it's not in the first 100 
a$app(visibility = "public", name = "bed converter")
## this return an app, because it pulls all apps and did serach.
a$app(visibility = "public", name = "bed converter", complete = TRUE)

Rate Limits

This call returns information about your current rate limit. This is the number of API calls you can make in one hour.

a$rate_limit()

Users

This call returns a list of the resources, such as projects, billing groups, and organizations, that are accessible to you. If you are not an administrator, this call will only return a successful response if {username} is replaced with your own username. If you are an administrator, you can replace {username} with the username of any CGC user, to return information on their resources.

Case sensitivity: Don’t forget to capitalize your username in the same way as you set it when you registered on the CGC.

If you don’t provide a username, your user information will be shown.

## return your information
a$user()
## return user 'tengfei''s information
a$user("tengfei")

Billing Group and Invoices

For billing

if no id provided, This call returns a list of paths used to access billing information via the API. else, This call lists all your billing groups, including groups that are pending or have been disabled. if breakdown = TRUE, This call returns a breakdown of spending per-project for the billing group specified by billing_group. For each project that the billing group is associated with, information is shown on the tasks run, including their initiating user (the runner), start and end times, and cost.

## return a BillingList object
(b <- a$billing())
a$billing(id = b$id, breakdown = TRUE)

For invoices

If no id provided, This call returns a list of invoices, with information about each, including whether or not the invoice is pending and the billing period it covers. The call returns information about all your available invoices, unless you use the query parameter bg_id to specify the ID of a particular billing group, in which case it will return the invoice incurred by that billing group only. if id provided, This call retrieves information about a selected invoice, including the costs for analysis and storage, and the invoice period.

a$invoice()
a$invoice(id = "fake_id")

Note (TODO): Invoice is not an object yet, it currently just return a list.

Project Operation

Project is the basic unit to organize different entities: files, tasks, apps, etc. So lots actions comes from this `Project’ object.

List All Projects

This call returns a list of all projects you are a member of. Each project’s project_id and URL on the CGC will be returned.

a$project()

Then if you want to list the projects owned by and accessible to a particular user, specify the owner argument. Each project’s ID and URL will be returned.

a$project(owner = "tengfei")
a$project(owner = "yintengfei")

To get details about project(s), use detail = TRUE

a$project(detail = TRUE)

Partial Match Project Name

For more friendly interface and convenient search, we support partial name match in this interface. The first argument for the call is “name”, users can provide part of the name and we do a search for you automatically.

## want to return a project called
a$project("hello")

Create a New Project

To create a new project, user need to specify

name (required)
billing_group_id (required)
description (optional)
tags (optional): this has to be a list(), only if you are “TCGA” user, you can create TCGA project by passing tags list(“tcga”)
type (optional): by default, we are creating a cwl project “v2”

a$project_new("api_testing_tcga", b$id,
              description = "Test for API")

Create a new project with TCGA controlled data on CGC

Just need to pass a “tags” list with value “tcga”

a$project_new("controlled_project", b$id,
              description = "Test for API", tags = list("tcga"))

Delete a Project

Next we delete what we created for testing, only single project could be deleted now by call $delete(), so please pay attention to the returned object from a$project(), sometimes if you are using partial matching by name, it will return a list. If you want to operate on a list of object, we provide some batch function, please read relevant section.

## remove it, not run
a$project("api_testing")$delete()
## check
## will delete all projects matcht the name
delete(a$project("api_testing_donnot_delete_me"))

Update/Edit a Project

You can update information about an existing project, including

name
description
billing_group

a$project(id = "tengfei/helloworld")
a$project(id = "tengfei/helloworld")$update(name = "Hello World Update", 
                                                    description = "Update description")

Project Member

List members

This call returns a list of the members of the specified project. For each member, the response lists:

The member’s username on the CGC
The member’s permissions in the project specified

a$project(id = "tengfei/demo-project")$member()

Add a member

This call adds a new user to a specified project. It can only be successfully made by a user who has admin permissions in the project.

Requests to add a project member must include the key permissions. However, if you do not include a value for some permission, it will be set to false by default.

Set permission by passing: copy, write, execute, admin, read argument.

Note: read is implicit and set by default, you can not be project member without having read permission

m <- a$project(id = "tengfei/demo-project")$member_add(username = "yintengfei")

Update a member

This call edits a user’s permissions in a specified project. It can only be successfully made by a user who has admin permissions in the project.

m <- a$project(id = "tengfei/demo-project")$
       member(username = "yintengfei")
m$update(copy = TRUE)

== Member ==
username : yintengfei
-- Permission --
read : TRUE
write : FALSE
copy_permission : TRUE
execute : FALSE
admin : FALSE

Delete a member

To delete an existing member, just to call delete() action on Member object.

m$delete()
## confirm
a$project(id = "tengfei/demo-project")$member()

List all Files

To list all files belongs to a project simple use

p <- a$project(id = "tengfei/demo-project")
p$file()

Files and Metadata

List all files

This call returns a list of all files in a specified project that you can access. For each file, the call returns:

Its ID
Its filename

The project is specified as a query parameter in the call.

a$file(project = p$id)
a$file("omni", project = p$id, detail = TRUE)

However we recommend user use cascading way to list files.

p$file()

To get details about files, please use detail = TRUE in the call.

## need to check
p$file(detail = TRUE)

Copy a file or group of files

This call copies the specified file to a new project. Files retain their metadata when copied, but may be assigned new names in their target project.

Note that Controlled Data files may not be copied to Open Data projects. To make this call, you should have copy permission within the project you are copying from.

Let’s try to copy a file from CGC public files, the id you can tell from the url is “561e1b33e4b0aa6ec48167d7”

You must provide

id file id, or list/vector of files ids.
project parameter: project id.
name is optional, if omitted, use the same.

## 1000G_omni2.5.b37.vcf
fid <- "561e1b33e4b0aa6ec48167d7"
fid2 <- "561e1b33e4b0aa6ec48167d3"
pid <- a$project("demo")$id
a$copyFile(c(fid, fid2), project = pid)
a$project(id = pid)$file()

NOTE: to copy a group of files, you need Auth$copyFile() interface. The id of those files in your project will be different from public id.

Alternatively you can do single file copy like this

a$project("hello")$file(id = fid)$copyTo(pid)

Delete file(s)

Note: the delete action only work for single file now, make sure your file call return a single file not a file list.

a$project("demo")$file()[[1]]$delete()
## confirm the deletion
a$project("demo")$file()

You can also delete a group of files or FilesList object, be careful with this function!

## return 5 files
a$project("demo")$file("phase1")
## delete all of them
delete(a$project("demo")$file("phase1"))
a$project("demo")$file("phase1")

Download files

To get the download information, basically a url, please use

a$project("demo")$file()[[1]]$download_url()

To download directly from R, use download call directly from single File object.

fid <- a$project("demo")$file()[[1]]$id
a$project("demo")$file(id = fid3)$download("~/Downloads/")

I also created download function for FilesList object to save your time

fls <- a$project("demo")$file()
download(fls, "~/Downloads/")

To download all files from a project.

a$project("demo")$download("~/Downloads")

Upload files

Seven Bridges platforms provide couple different ways for data import

command line uploader
graphic UI uploader
from ftp, http etc from interface directly
api uploader that you can directly call with sevenbridges package

API client uploader is working like this, simply call project$upload function to upload a file a file list or a folder recursively…

a <- Auth(username = "tengfei", platform = "cgc")
fl <- system.file("extdata", "sample1.fastq", package = "sevenbridges")

(p <- a$project(id = "tengfei/quickstart"))

## by default load .meta for the file
p$upload(fl, overwrite = TRUE)
## pass metadata
p$upload(fl, overwrite = TRUE, metadata = list(library_id = "testid2", platform = "Illumina x11"))
## rename
p$upload(fl, overwrite = TRUE, name = "sample_new_name.fastq", 
         metadata = list(library_id = "new_id"))

Upload a folder

dir.ext <- system.file("extdata", package = "sevenbridges")
list.files(dir.ext)
p$upload(dir.ext, overwrite = TRUE)

Upload a file list

dir.ext <- system.file("extdata", package = "sevenbridges")
## enable full name
fls <- list.files(dir.ext, recursive = TRUE, full.names = TRUE)
p$upload(fls, overwrite = TRUE)
p$upload("~/Documents/Data/sbgtest/1000G_phase1.snps.high_confidence.b37.vcf")

Files filter from a task

You can also get all files from a task, or by a metadata filter.

## list all outputs file from a task id
a$task(id = "53020538-6936-422f-80de-02fa65ae4b39")$file()

## alternative way to list files under specific project
a$file(project = "tengfei/re",
       origin.task = "53020538-6936-422f-80de-02fa65ae4b39")


## you can filter by metadata as well
a$file(project = "tengfei/re", 
       origin.task = "53020538-6936-422f-80de-02fa65ae4b39", 
       metadata = list(experimental_strategy = "RNA-Seq"))

Public files

The only way you can list public files now via API is from Auth$file() call, and for now, the project id is “admin/sbg-public-data”. This may be updated later. Alternative, just click the file from our GUI, you will see the id in the url link.

a$file(project = "admin/sbg-public-data")

Update a file

You can call update() function from Files object, following things could be updated

name
metadata (list): this is going to overwrite all meta for the file, so please provide the full list. For more flexible operation, please check next section about Metadata.

If no parameters provided, will just get detail for the same file and update the object itself.

(fl <- a$project(id = "tengfei/demo-project")$file(name = "sample.fastq"))

== File ==
id : 56c7916ae4b03b56a7d7
name : sample.fastq
project : tengfei/demo-project

Show metadata

## show metadata
fl$meta()

Update meta

fl$update(name = "sample.fastq", metadata = list(new_item1 = "item1", new_item2 = "item2", file_extension = "fastq"))
## check it out
fl$meta()

Metadata Operation

A full list of metadata fields and their permissible values on the CGC is available on the page TCGA Metadata.

Note that the file name is not the same as its ID. The ID is a hexadecimal string, automatically assigned to a file in a project. The file’s name is a human-readable string. For information, please see the API overview.

To get metadata for a file call meta().

## meta is pulling the latest information via API
fl$meta()
## field meta data saved the previous saved one
fl$metadata

Although CGC defined a set of meta schema, which is visible on the UI of the platform, but you can pass any free form of meta for the file, it’s just not visible on UI, but it’s stored with the data.

Only the value being specified stored with files, to set metadata please call setMeta() from Files object.

Important:

By default, we are not overwriting the meta field using setMeta call, unless you pass the overwrite = TRUE argument

fl$setMeta(new_item3 = "item3")
fl
## oops it removed rest of the meta
fl$setMeta(new_item4 = "item4", overwrite = TRUE)
fl

Let’s keep playing with meta, if you are really interested in the default schema that shown on the UI, you can use Metadata() constructor and check details of each meta; Simply call the function (name of meta), it will show description and enumerated items. Please pay attention to suggested_values field.

## check which schema we have
Metadata()$show(full = TRUE)
## check details for each, play with it
platform()
paired_end()
quality_scale()

You can see some have suggested value, to construct the Metadata, we encourage you use Metadata() directly, pass metadata directly into the call, it will do the validation.

Metadata(platform  = "Affymetrix SNP Array 6.0", 
         paired_end = 1, 
         quality_scale = "sanger",
         new_item = "new test")

App

From now on we are going to have fun with Apps! The CWL(Common Workflow Language) based approach. It gets more and more popular and really designed for reproducible pipeline description and execution. All Seven Bridges platforms support cwl naively in the cloud. So in this section, I will introduce how we are going to do this via API and inside R.

List all apps

This call lists all the apps available to you.

a$app()
## or show details
a$app(detail = TRUE)

To search a name, please pass a pattern for the name argument; or provide a unique id.

## pattern match
a$app(name = "STAR")
## unique id
aid <- a$app()[[1]]$id
aid
a$app(id = aid)
## get a specific revision from an app
a$app(id = aid, revision = 0)

To list all apps belong to one project use project argument

## my favorite, always
a$project("demo")$app()

## or alternatviely
pid <- a$project("demo")$id
a$app(project = pid)

To list all public apps, use visibility argument

## show 100 items from public
x = a$app(visibility = "public")
length(x)
x = a$app(visibility = "public", complete = TRUE)
length(x)
x = a$app(project = "tengfei/helloworld", complete = TRUE)
length(x)
a$app(visibility = "public", limit = 5, offset = 150)

To search an app cross all published apps (this may take a while)

a$app("STAR", visibility = "public", complete = TRUE)

Copy an App

This call copies the specified app to the specified project. The app should be one in a project that you can access; this could be an app that has been uploaded to the CGC by a project member, or a publicly available app that has been copied to the project.

Need two arguments

project: id character
name: optional, to re-name your app

aid <- a$app(visibility = "public")[[1]]$id
a$copyApp(aid, project = pid, name = "copy-rename-test")
## check it is copied
a$app(project = pid)

Get CWL from an App

This call returns information about the specified app, as raw CWL. The call differs from the call to GET details of an app by returning a JSON object that is the CWL.

The app should be one in a project that you can access; this could be an app that has been uploaded to the CGC by a project member, or a publicly available app that has been copied to the project.

To get a specific revision, pass revision argument.

ap <- a$app(visibility = "public")[[1]]
a$project("demo")$app("index")
## get a specific revision
a$project("demo")$app("index", revision = 0)

TODO: convert it to an CWL object

Add CWL as an APP

Use app_add function call from a Project object, two parameters required

short_name: a short id for your app, alphanumeric character, no spacing; this not name field.
filename: you json file for cwl.

cwl.fl <- system.file("extdata", "bam_index.json", package = "sevenbridges")
a$project("demo")$app_add(short_name = "new_bam_index_app", filename = cwl.fl)
a$project("demo")$app_add(short_name = "new_bam_index_app", revision = 2, filename = cwl.fl)

Note: provide the same short_name will add new revision

Directly Describe CWL in R

This is fun and is introduced in another vignette.

Task Operation

List tasks

This call returns a list of tasks that you can access. You are able to filter by status

## all tasks
a$task()
## filter
a$task(status = "completed")
a$task(status = "running")

To list all tasks in a project

## better way
a$project("demo")$task()

## alternatively
pid <- a$project("demo")$id
pid
a$task(project = pid)

Create a draft task

To create a draft, you need to call the task_add function from Project object. And you need to pass following arguments

name: name for this task
description: description for this task
app: app id you have access to
inputs: inputs list for this task

## push an app first
fl.runif <- system.file("extdata", "runif.json", package = "sbgr")
a$project("demo")$app_add("runif_draft", fl.runif)
runif_id <- "tengfei/demo-project/runif_draft"
## create a draft task
a$project("demo")$task_add(name = "Draft runif 3", 
                           description = "Description for runif 3", 
                           app = runif_id,
                           inputs = list(min = 1, max = 10))
## confirm
a$project("demo")$task(status = "draft")

Modify a task

Call update function fro a Task object, you can update

name
description
inputs list (only update items you provided.)

## get the single task you want to update
tsk <- a$project("demo")$task("Draft runif 3")
tsk
tsk$update(name = "Draft runif update", description = "draft 2", 
           inputs = list(max = 100))
## alternative way to check all inputs
tsk$getInputs()

Run a task

This call runs (executes) the specified task. Only tasks whose status is “DRAFT” may be run.

tsk$run()
## run update without information just return latest information
tsk$update()

Monitor a running task and set function hook

To monitor a running task, call monitor from a task object

first argument set interval time to check the status
rest arguments might be used for hook function

tsk$monitor()

get and set default hook function for task status, currently failed, completed tasks will break the monitoring.

Note: Hook function has to return TRUE (break monitoring) or FALSE (continuing) in the end.

getTaskHook("completed")
getTaskHook("draft")
setTaskHook("draft", function(){message("never happens"); return(TRUE)})
getTaskHook("draft")

Abort a runing task

This call aborts the specified task. Only tasks whose status is “RUNNING” may be aborted.

## abort
tsk$abort()
## check
tsk$update()

Delete a task

Note that you can only delete draft tasks, not running tasks.

tsklst <- a$task(status = "draft")
## delete a single task
tsklst[[1]]$delete()
## confirm
a$task(status = "draft")
## delete a list of tasks
delete(tsklst)

Download all files from a completed task

tsk$download("~/Downloads")

Run task in bacth mode

To run task in batch mode, (check ?batch) for more details, here is an mock running

## batch by items
(tsk <- p$task_add(name = "RNA DE report new batch 2", 
                   description = "RNA DE analysis report", 
                   app = rna.app$id,
                   batch = batch(input = "bamfiles"),
                   inputs = list(bamfiles = bamfiles.in, 
                                 design = design.in,
                                 gtffile = gtf.in)))

## batch by metadata, input files has to have metadata fields specified
(tsk <- p$task_add(name = "RNA DE report new batch 3", 
                   description = "RNA DE analysis report", 
                   app = rna.app$id,
                   batch = batch(input = "fastq", 
                                 c("metadata.sample_id", "metadata.library_id")),
                   inputs = list(bamfiles = bamfiles.in, 
                                 design = design.in,
                                 gtffile = gtf.in)))

Get Raw Response from httr

In easy API, we return an object which contains the raw response from httr as a field, you can either call response() on that object or just get the field out of it

Batch operation on project/files/tasks

Right now, users have to use lapply to do those operations themselves. It’s simple implementation.

In this package, we implement delete and download for some object like task and project or file.