RESTful Guide with curl and Store filenames with emoticons: Difference between pages

From LogicalDOC Community Wiki
(Difference between pages)
Jump to navigationJump to search
Added enterpriseSearch example
 
Giuseppe (talk | contribs)
192 edits
No edit summary
 
Line 1: Line 1:
{{TOCright}} __TOC__
= Preparing a Dataset with Label Studio =


{{Note|If you're interested in use LogicalDOC API we suggest to '''take a look at ours [[Bindings_And_Samples|Bindings and Samples]] and take advantage of already usable examples which connect to webservices API.}}
This guide explains how to create an annotated dataset for YOLO training using Label Studio.


LogicalDOC has a [http://docs.logicaldoc.com/en/web-services-api complete API exposed via REST]. This means you can call any of these API methods from any programming language, like Java, PHP or Python among others. This feature makes it possible to create a custom client, or integrate with third-party applications like a CRM or a CMS.
=== Install Label Studio ===


{{Advice|The examples in this article may not work with your LogicalDOC; the [http://docs.logicaldoc.com/en/web-services-api REST API] is currently in development, so expect changes and additions.}}
Refer to the official installation guide:
https://labelstud.io/guide/install


If you point your browser to http://localhost:8080/services, you can see the SOAP API at first place but at the bottom you will see a '''Available RESTful services''' section. These URLs are protected by BASIC authentication so you need to provide an user and password to access them.
=== Enable Local File Storage ===


== Sample usage ==
For large projects it is not recommended to upload images directly through the Label Studio interface.
To try these API methods you can use an HTTP Client library or any REST client which ease this process. Or simply you can use the '''curl''' command-line application. For example, you can list the children folders:


<syntaxhighlight lang="bash">
To enable local file access, configure the following environment variables:
  $ curl -u admin:admin -H "Accept: application/json" \
    http://localhost:8080/services/rest/folder/listChildren?folderId=4
</syntaxhighlight>
 
The result is:


<pre>
<pre>
[
LABEL_STUDIO_LOCAL_FILES_SERVING_ENABLED=true
  {
LABEL_STUDIO_LOCAL_FILES_DOCUMENT_ROOT=/path/to/images
    "id": 3440640,
    "name": "alfa",
    "parentId": 4,
    "description": "",
    "lastModified": "2016-06-15 15:49:40 +0200",
    "type": 0,
    "templateId": null,
    "templateLocked": 0,
    "creation": "2016-06-15 15:49:40 +0200",
    "creator": "Admin Admin",
    "position": 1,
    "hidden": 0,
    "foldRef": null,
    "attributes": [
     
    ]
  },
  {
    "id": 3440643,
    "name": "beta",
    "parentId": 4,
    "description": "",
    "lastModified": "2016-06-16 10:16:25 +0200",
    "type": 0,
    "templateId": null,
    "templateLocked": 0,
    "creation": "2016-06-16 09:49:27 +0200",
    "creator": "Admin Admin",
    "position": 1,
    "hidden": 0,
    "foldRef": null,
    "attributes": [
     
    ]
  }
]
</pre>
</pre>


In this case you can see the result in JSON format.
Launch Label Studio:
Some endpoints can also provide the results in XML format but you have to check them, if that is supported we can make a call sending the appropriate '''Accept''' header:


<syntaxhighlight lang="bash">
<pre>
  $ curl -u admin:admin -H "Accept: application/xml" \
label-studio start
    http://localhost:8080/services/rest/folder/listChildren?folderId=4
</pre>
</syntaxhighlight>
 
The result in XML is:
 
<syntaxhighlight lang="xml">
<?xml version="1.0" encoding="UTF-8"?>
<folders>
  <folder>
    <creation>2016-06-15 15:49:40 +0200</creation>
    <creator>Admin Admin</creator>
    <description></description>
    <hidden>0</hidden>
    <id>3440640</id>
    <lastModified>2016-06-15 15:49:40 +0200</lastModified>
    <name>alfa</name>
    <parentId>4</parentId>
    <position>1</position>
    <templateLocked>0</templateLocked>
    <type>0</type>
  </folder>
  <folder>
    <creation>2016-06-16 09:49:27 +0200</creation>
    <creator>Admin Admin</creator>
    <description></description>
    <hidden>0</hidden>
    <id>3440643</id>
    <lastModified>2016-06-16 10:16:25 +0200</lastModified>
    <name>beta</name>
    <parentId>4</parentId>
    <position>1</position>
    <templateLocked>0</templateLocked>
    <type>0</type>
  </folder>
</folders>
</syntaxhighlight>
 
This is a Java client for the same call:
 
<syntaxhighlight lang="java">
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.Authenticator;
import java.net.HttpURLConnection;
import java.net.MalformedURLException;
import java.net.PasswordAuthentication;
import java.net.URL;
 
public class JavaRestClient {
    public static void main(String[] args) throws Exception {
        try {
            long folderID = 4L;
            URL url = new URL("http://localhost:8080/services/rest/folder/listChildren?folderId=" + folderID);
            HttpURLConnection conn = (HttpURLConnection) url.openConnection();
            conn.setRequestMethod("GET");
            conn.setRequestProperty("Accept", "application/json");
           
            Authenticator.setDefault(new Authenticator() {
                protected PasswordAuthentication getPasswordAuthentication() {
                    return new PasswordAuthentication("admin", "admin".toCharArray());
                }
            });
           
            if (conn.getResponseCode() == 200) {
                BufferedReader br = new BufferedReader(new InputStreamReader((conn.getInputStream())));
                System.out.println("Output from Server .... \n");
                String output;
               
                while ((output = br.readLine()) != null) {
                    System.out.println(output);
                }
            } else {
                System.err.println("Failed : HTTP error code : " + conn.getResponseCode());
            }
           
            conn.disconnect();
        } catch (MalformedURLException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}
</syntaxhighlight>
 
== Folder ==
Let's create a new folder:
 
<syntaxhighlight lang="bash">
  $ curl -u admin:admin -H "Accept: application/json" \
    -X POST -H "Content-Type: text/plain" -d "/Default/Curl/newfolder" \
    http://localhost:8080/services/rest/folder/createSimple
</syntaxhighlight>
 
Creates a path of folders starting from the folder with ID 4 (Default folder)
<syntaxhighlight lang="bash">
  $ curl -u admin:admin -H "Accept: application/json" \
    -X POST -H "Content-Type: application/x-www-form-urlencoded" -d parentId=4 -d path=How/to/POST/JSON/data/with/Curl \
    http://localhost:8080/services/rest/folder/createPath
</syntaxhighlight>
 
== Document ==
 
=== Create a Document ===
Now we are going to create a document. For this, we need to provide the document binary data:
 
<syntaxhighlight lang="bash">
  $ curl -u admin:admin -H "Accept: application/json" \
    -X POST -F folderId=4 -F filename=CHANGELOG.txt -F filedata=@CHANGELOG.txt \
    http://localhost:8080/services/rest/document/upload
</syntaxhighlight>
 
In this case the document will be added to the respository using the default language (english). Of course it is possible to specify the additional parameter 'language' to tell the system that the document we are storing is in german (ISO 639-2 code)
 
<syntaxhighlight lang="bash">
  $ curl -u admin:admin -H "Accept: application/json" \
    -X POST -F folderId=4 -F filename=pub_arbeitsplatz_straße.pdf -F language=de -F filedata=@pub_arbeitsplatz_straße.pdf \
    http://localhost:8080/services/rest/document/upload
</syntaxhighlight>
 
Creates a document with the create method on Windows 10<br/>
Windows 10, curl 7.55.1 (Windows) libcurl/7.55.1 WinSSL
 
<syntaxhighlight lang="bash">
  $ curl -v -u admin:admin -X POST "http://localhost:8080/services/rest/document/create" -H "accept: application/json" -H "Content-Type: multipart/form-data" -F "document={ \"language\":\"en\",\"fileName\":\"ScreenHunter949.png\",\"folderId\":4 };type=application/json" -F "content=@C:\Users\shatz\Desktop\ScreenHunter949.png;type=application/octet-stream"
</syntaxhighlight>
 
Or also from a HTML form:
 
<syntaxhighlight lang="html4strict">
<html>
  <body>
    <form method="POST" enctype="multipart/form-data"
          action="http://localhost:8080/services/rest/document/upload">
      Select folder: <input type="text" name="folderId" value="4"/><br/>
      Select filename: <input type="text" name="filename" /><br/>
      Select file: <input type="file" name="filedata" size="45"/><br/>
      <input type="submit" value="Upload" />
    </form>
  </body>
</html>
</syntaxhighlight>
 
=== Create via Upload ===
Create a new document, or a new version of an existing document (in one step) [Windows 11]<br/>
This is a simpler way of creating documents, however it is more limited as it does not allow complete control of the metadata
 
<syntaxhighlight lang="bash">
  $ curl -u admin:admin -X POST "http://localhost:8080/services/rest/document/upload" \
    -H  "Accept: application/json" -H  "Content-Type: multipart/form-data" -F "docId=721" -F "folderId=" -F "release=false" \
    -F "filename=logicaldoc_community - Checkmarx AST.pdf" -F "language=en" -F "filedata=@logicaldoc_community - Checkmarx AST.pdf;type=application/pdf"
</syntaxhighlight>


Upload: updates the content of an existing document generating a new version
=== Create a Project ===
<syntaxhighlight lang="bash">
  $ curl -v -u admin:admin -H "Accept: application/json" -X POST -F docId=118 -F filename=google.png -F filedata=@C:\tmp\google.png http://localhost:8080/services/rest/document/upload
</syntaxhighlight>


=== Update metadata ===
# Login to Label Studio
Update the document metadata. Specifically, we can see how to update an extended attribute field of type date (type = 3) using the property dateValue
# Click '''Create Project'''
# Enter a project name
# Configure the labeling interface
# Save the project


<syntaxhighlight lang="bash">
=== Import Images ===
  $ curl -v -u admin:admin -H "Content-Type: application/json" -H "Accept: application/json" -X PUT \
    -d "{ \"id\": 47, \"folderId\": 4, \"fileName\":\"Egzai_u002.doc\", \"templateId\":92241920, \"attributes\":[{\"name\":\"ack\",\"stringValue\":\"ack\",\"type\":0},{\"name\":\"Tar\",\"dateValue\":\"2017-03-18 19:10:00 +0100\",\"type\":3}] }" \
    http://localhost:8080/services/rest/document/update
</syntaxhighlight>


=== Delete ===
# Open the project
Delete a specific version of a given document (since v7.6.4)
# Click '''Import'''
# Select '''Local Storage'''


<syntaxhighlight lang="bash">
When importing images, choose '''Files''' as the import method.
  $ curl -u admin:admin \
    -G -d docId=1803 -d version=1.3 -X DELETE http://localhost:8080/services/rest/document/deleteVersion
</syntaxhighlight>


=== Download ===
Unlike CVAT, Label Studio creates one task for each imported document image.


And now download it:
[[File:LabelStudio-import-method.png|thumb|600px|center|Selecting the Files import method]]


<syntaxhighlight lang="bash">
=== Annotate Documents ===
  $ curl -u admin:admin \
    http://localhost:8080/services/rest/document/getContent?docId=456456456
</syntaxhighlight>


If the document is a binary file you can redirect the output to a file adding '> filename' to the end of the command
# Open a task
<syntaxhighlight lang="bash">
# Select a label
  $ curl -u admin:admin \
# Draw a bounding box around the target area
    http://localhost:8080/services/rest/document/getContent?docId=456456456 > myFile.pdf
# Save the annotation
</syntaxhighlight>


=== Convert a document to PDF ===
Example labels:


Note: This will only perform the conversion, if completed successfully it returns HTTP code 204 and empty content
* Invoice Number
<syntaxhighlight lang="bash">
* Date
  $ curl -v -X PUT "http://localhost:8080/services/rest/document/createPdf?docId=125" -H  "accept: application/json" -H  "Authorization: Basic YWRtaW46YWRtaW4="
* Seller Name
</syntaxhighlight>
* Buyer Name
* Total Amount


Download the converted .PDF file (latest version of the document)
[[File:LabelStudio-annotation-example.png|thumb|600px|center|Example annotation]]
<syntaxhighlight lang="bash">
  $ curl -v -X GET "http://localhost:8080/services/rest/document/getResource?docId=125&suffix=conversion.pdf" -H  "accept: application/octet-stream" -H  "Authorization: Basic YWRtaW46YWRtaW4=" --output 125-conversion.pdf
</syntaxhighlight>
 
=== Versioning ===
Checkout an existing document<br/>
 
<syntaxhighlight lang="bash">
  $ curl -k -u admin:admin -X POST https://[url]/services/rest/document/checkout \
    -H  "accept: application/json" -H  "Content-Type: application/x-www-form-urlencoded" -d "docId=2686"
</syntaxhighlight>
 
Checkin a new version of document (the document must be in checked-out state) [Windows 11]<br/>
<syntaxhighlight lang="bash">
  $ curl -v -u admin:admin -X POST "http://localhost:8080/services/rest/document/checkin" \
    -H  "accept: */*" -H  "Content-Type: multipart/form-data" -F "docId=721" -F "comment=" -F "release=false" -F "filename=Checkmarx.txt" -F "filedata=@Checkmarx.txt;type=text/plain"
</syntaxhighlight>
 
== Search ==
Standard Full-text search on content, title and tags using english as language of the query (expressionLanguage) and on english documents (language):
 
<syntaxhighlight lang="bash">
  $ curl -u admin:admin -H "Content-Type: application/json" -H "Accept: application/json" -X POST \
    -d "{\"maxHits\":50,\"expression\":\"document management system\",\"expressionLanguage\":\"en\",\"language\":\"en\", \"dateFrom\": \"2023-01-01\"}"
    http://localhost:8080/services/rest/search/find
</syntaxhighlight>
 
Date fields you can use:
<pre>
"dateFrom": "string",
"dateTo": "string",
"creationFrom": "string",
"creationTo": "string",
</pre>


dates must be expressed in ISO format: yyyy-mm-dd or yyyy-MM-dd HH:mm:ss (24 hours)
=== Export the Dataset ===


== Tags ==
# Open the project
Set the document tags of document with ID = 325
# Click '''Export'''
# Select the desired format


<syntaxhighlight lang="bash">
Supported formats include:
  $ curl -v -u admin:admin -X POST "http://localhost:8080/services/rest/tag/setDocumentTags" \
        -H "accept: application/json" -H "Content-Type: application/x-www-form-urlencoded" \
        -d "docId=325" --data-urlencode "tag=fileExtension:docx" \
        --data-urlencode "tag=wopiLockString:1234567890123456789012345678901234567890123456789012345678901234567890"
</syntaxhighlight>


== Upload Resource ==
* YOLO
* COCO
* Pascal VOC
* CSV


Uploads a resource of an existing document: thumbnail, tile, or preview.<br/>
For YOLO training, export the dataset in YOLO format.
The following example uploads the tile image (suffix: tile.png)


<syntaxhighlight lang="bash">
=== Dataset Formats ===
curl -X "POST" ^
  "http://localhost:8080/services/rest/document/uploadResource" ^
  -H "accept: */*" ^
  -H "Authorization: Basic YWRtaW46YWRtaW4=" ^
  -H "Content-Type: multipart/form-data" ^
  -F "docId=10075" ^
  -F "fileVersion=1.0" ^
  -F "suffix=tile.png" ^
  -F "content=@folder32.png;type=image/png"
</syntaxhighlight>


== Enterprise Methods ==
==== COCO ====
Additional search methods available in commercial versions of LogicalDOC


=== Enterprise Searches ===
COCO is a JSON-based dataset format commonly used for object detection datasets.


This call will query the document table and count the documents in folder: /Default (folder ID = 4)
More information:
https://docs.aws.amazon.com/rekognition/latest/customlabels-dg/md-coco-overview.html


<syntaxhighlight lang="bash">
==== YOLO ====
  $ curl -X POST "http://localhost:8080/services/rest/enterpriseSearch/queryDocuments" \
              -H "accept: application/json" -H "Authorization: Basic YWRtaW46YWRtaW4=" -H  "Content-Type: application/x-www-form-urlencoded" \
              -d "select=COUNT(*)&where=DOC.ld_folderid%3D4&maxResults=2000000"
</syntaxhighlight>


YOLO datasets contain images and annotation files organized according to a predefined directory structure.


More info at:
More information:
https://docs.cvat.ai/docs/dataset_management/formats/format-yolo/


* [http://www.yilmazhuseyin.com/blog/dev/curl-tutorial-examples-usage/ curl tutorial with examples of usage]
==== YOLOv8 OBB ====
* [http://docs.logicaldoc.com/resources/wsdoc/rest/index.html?version=7.6.4 LogicalDOC REST API reference v7.6.4]
* [https://docs.logicaldoc.com/resources/wsdoc/rest/index.html?version=7.7.6 LogicalDOC REST API reference v7.7.6]
* [https://docs.logicaldoc.com/resources/wsdoc/rest/index.html?version=8.1.0 LogicalDOC REST API reference v8.1]
* [https://docs.logicaldoc.com/resources/wsdoc/rest/9.1.1/index.html LogicalDOC REST API reference v9.1.1]
* [https://app.swaggerhub.com/apis/swatzniak/logicaldoc_rest_api/8.8.1 LogicalDOC REST API | 8.8.1 | SwaggerHub]
* [[Bindings_And_Samples|Webservices - Binding and Examples]]
* [https://docs.logicaldoc.com/en/web-services-api LogicalDOC Web Services API]


[[Category: RESTful Guide]]
YOLOv8 OBB (Oriented Bounding Boxes) extends the standard YOLO format by supporting rotated bounding boxes using eight normalized coordinates.

Revision as of 12:23, 23 June 2026

Preparing a Dataset with Label Studio

This guide explains how to create an annotated dataset for YOLO training using Label Studio.

Install Label Studio

Refer to the official installation guide: https://labelstud.io/guide/install

Enable Local File Storage

For large projects it is not recommended to upload images directly through the Label Studio interface.

To enable local file access, configure the following environment variables: 

LABEL_STUDIO_LOCAL_FILES_SERVING_ENABLED=true
LABEL_STUDIO_LOCAL_FILES_DOCUMENT_ROOT=/path/to/images

Launch Label Studio: 

label-studio start

Create a Project

  1. Login to Label Studio
  2. Click Create Project
  3. Enter a project name
  4. Configure the labeling interface
  5. Save the project

Import Images

  1. Open the project
  2. Click Import
  3. Select Local Storage

When importing images, choose Files as the import method.

Unlike CVAT, Label Studio creates one task for each imported document image.

File:LabelStudio-import-method.png
Selecting the Files import method

Annotate Documents

  1. Open a task
  2. Select a label
  3. Draw a bounding box around the target area
  4. Save the annotation

Example labels:

  • Invoice Number
  • Date
  • Seller Name
  • Buyer Name
  • Total Amount
File:LabelStudio-annotation-example.png
Example annotation

Export the Dataset

  1. Open the project
  2. Click Export
  3. Select the desired format

Supported formats include:

  • YOLO
  • COCO
  • Pascal VOC
  • CSV

For YOLO training, export the dataset in YOLO format.

Dataset Formats

COCO

COCO is a JSON-based dataset format commonly used for object detection datasets.

More information: https://docs.aws.amazon.com/rekognition/latest/customlabels-dg/md-coco-overview.html

YOLO

YOLO datasets contain images and annotation files organized according to a predefined directory structure.

More information: https://docs.cvat.ai/docs/dataset_management/formats/format-yolo/

YOLOv8 OBB

YOLOv8 OBB (Oriented Bounding Boxes) extends the standard YOLO format by supporting rotated bounding boxes using eight normalized coordinates.

Retrieved from "http://wiki.logicaldoc.com/wiki/index.php?title=RESTful_Guide_with_curl&oldid=5347"