Line 1:
Line 1: {{TOCright}} __TOC__
= Preparing a Dataset with Label Studio =
{{Note|If you're interested in use LogicalDOC API we suggest to '''take a look at ours [[Bindings_And_Samples|Bindings and Samples]] and take advantage of already usable examples which connect to webservices API.}}
This guide explains how to create an annotated dataset for YOLO training using Label Studio .
LogicalDOC has a [http://docs.logicaldoc.com/en/web-services-api complete API exposed via REST]. This means you can call any of these API methods from any programming language, like Java, PHP or Python among others. This feature makes it possible to create a custom client, or integrate with third-party applications like a CRM or a CMS.
=== Install Label Studio ===
{{Advice|Examples in this page refer to LogicalDOC 7.7; the [http://docs.logicaldoc.com/en/web-services-api REST API] is currently in development, so expect changes and additions.}}
Refer to the official installation guide:
https ://labelstud .io /guide /install
If you point your browser to http://localhost:8080/services, you can see the SOAP API at first place but at the bottom you will see a '''Available RESTful services''' section. These URLs are protected by BASIC authentication so you need to provide an user and password to access them.
=== Enable Local File Storage ===
== Sample usage ==
For large projects it is not recommended to upload images directly through the Label Studio interface .
To try these API methods you can use an HTTP Client library or any REST client which ease this process. Or simply you can use the '''curl''' command-line application. For example, you can list the children folders:
$ curl -u admin:admin -H "Accept: application/json" \
To enable local file access, configure the following environment variables :
http://localhost:8080/services/rest/folder/listChildren?folderId=4
The result is:
<pre>
LABEL_STUDIO_LOCAL_FILES_SERVING_ENABLED=true
LABEL_STUDIO_LOCAL_FILES_DOCUMENT_ROOT=/path/to/images
</pre>
<source lang="text">
Launch Label Studio :
[
{
"id": 3440640,
"name": "alfa",
"parentId": 4,
"description": "",
"lastModified": "2016-06-15 15:49:40 +0200",
"type": 0,
"templateId": null,
"templateLocked": 0,
"creation": "2016-06-15 15:49:40 +0200",
"creator": "Admin Admin",
"position": 1,
"hidden": 0,
"foldRef": null,
"attributes": [
]
},
{
"id": 3440643,
"name": "beta",
"parentId": 4,
"description": "",
"lastModified": "2016-06-16 10:16:25 +0200",
"type": 0,
"templateId": null,
"templateLocked": 0,
"creation": "2016-06-16 09:49:27 +0200",
"creator": "Admin Admin",
"position": 1,
"hidden": 0,
"foldRef": null,
"attributes": [
]
}
]
</source>
In this case you can see the result in JSON format.
<pre >
Some endpoints can also provide the results in XML format but you have to check them, if that is supported we can make a call sending the appropriate '''Accept''' header:
label -studio start
</pre >
$ curl -u admin:admin -H "Accept: application/xml" \
http://localhost:8080/services/rest/folder/listChildren?folderId=4
The result in XML is:
<source lang="xml">
<?xml version="1.0" encoding="UTF-8"?>
<folders>
<folder>
<creation>2016-06-15 15:49:40 +0200</creation>
<creator>Admin Admin</creator>
<description></description>
<hidden>0</hidden>
<id>3440640</id>
<lastModified>2016-06-15 15:49:40 +0200</lastModified>
<name>alfa</name>
<parentId>4</parentId>
<position>1</position>
<templateLocked>0</templateLocked>
<type>0</type>
</folder>
<folder>
<creation>2016-06-16 09:49:27 +0200</creation>
<creator>Admin Admin</creator>
<description></description>
<hidden>0</hidden>
<id>3440643</id>
<lastModified>2016-06-16 10:16:25 +0200</lastModified>
<name>beta</name>
<parentId>4</parentId>
<position>1</position>
<templateLocked>0</templateLocked>
<type>0</type>
</folder>
</folders>
</source>
This is a Java client for the same call:
<source lang="java">
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.Authenticator;
import java.net.HttpURLConnection;
import java.net.MalformedURLException;
import java.net.PasswordAuthentication;
import java.net.URL;
public class JavaRestClient {
public static void main(String[] args) throws Exception {
try {
long folderID = 4L;
URL url = new URL("http://localhost:8080/services/rest/folder/listChildren?folderId=" + folderID);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("GET");
conn.setRequestProperty("Accept", "application/json");
Authenticator.setDefault(new Authenticator() {
protected PasswordAuthentication getPasswordAuthentication() {
return new PasswordAuthentication("admin", "admin".toCharArray());
}
});
if (conn.getResponseCode() == 200) {
BufferedReader br = new BufferedReader(new InputStreamReader((conn.getInputStream())));
System.out.println("Output from Server .... \n");
String output;
while ((output = br.readLine()) != null) {
System.out.println(output);
}
} else {
System.err.println("Failed : HTTP error code : " + conn.getResponseCode());
}
conn.disconnect();
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
}
</source>
== Folder ==
Let's create a new folder:
$ curl -u admin:admin -H "Accept: application/json" \
-X POST -H "Content-Type: text/plain" -d "/Default/Curl/newfolder" \
http://localhost:8080/services/rest/folder/createSimple
Creates a path of folders starting from the folder with ID 4 (Default folder)
$ curl -u admin:admin -H "Accept: application/json" \
-X POST -H "Content-Type: application/x-www-form-urlencoded" -d parentId=4 -d path=How/to/POST/JSON/data/with/Curl \
http://localhost:8080/services/rest/folder/createPath
== Document ==
=== Create a Project = ==
Now we are going to create a document. For this, we need to provide the document binary data:
$ curl -u admin:admin -H "Accept: application/json" \
# Login to Label Studio
-X POST -F folderId=4 -F filename=CHANGELOG.txt -F filedata=@CHANGELOG.txt \
# Click '''Create Project'''
http://localhost:8080/services/rest/document/upload
# Enter a project name
# Configure the labeling interface
# Save the project
In this case the document will be added to the respository using the default language (english). Of course it is possible to specify the additional parameter 'language' to tell the system that the document we are storing is in german (ISO 639-2 code)
=== Import Images ===
$ curl -u admin:admin -H "Accept: application/json" \
# Open the project
-X POST -F folderId=4 -F filename=pub_arbeitsplatz_straße.pdf -F language=de -F filedata=@pub_arbeitsplatz_straße.pdf \
# Click '''Import'''
http://localhost:8080/services/rest/document/upload
# Select '''Local Storage'''
When importing images, choose '''Files''' as the import method.
Or also from a HTML form:
Unlike CVAT, Label Studio creates one task for each imported document image.
<source lang="html4strict">
[[File:LabelStudio-import- method.png|thumb|600px|center|Selecting the Files import method]]
<html>
<body>
<form method="POST" enctype="multipart/form-data"
action="http://localhost:8080/services/rest/document/upload">
Select folder: <input type="text" name="folderId" value="4"/><br/>
Select filename: <input type="text" name="filename" /><br/>
Select file: <input type="file" name="filedata" size="45"/><br/>
<input type="submit" value="Upload" />
</form>
</body>
</html>
</source>
And now download it:
=== Annotate Documents ===
$ curl -u admin:admin \
# Open a task
http://localhost:8080/services/rest/document/getContent?docId=456456456
# Select a label
# Draw a bounding box around the target area
# Save the annotation
If the document is a binary file you can redirect the output to a file adding '> filename' to the end of the command
Example labels:
$ curl -u admin:admin \
* Invoice Number
http://localhost:8080/services/rest/document/getContent?docId=456456456 > myFile.pdf
* Date
* Seller Name
* Buyer Name
* Total Amount
Delete a specific version of a given document (since v7.6.4)
[[File:LabelStudio-annotation-example .png|thumb|600px|center|Example annotation]]
$ curl -u admin:admin \
=== Export the Dataset ===
-G -d docId=1803 -d version=1.3 -X DELETE http://localhost:8080/services/rest/document/deleteVersion
Update the document metadata. Specifically, we can see how to update an extended attribute field of type date (type = 3) using the property dateValue
# Open the project
# Click '''Export'''
# Select the desired format
$ curl -v -u admin:admin -H "Content-Type: application/json" -H "Accept: application/json" -X PUT \
Supported formats include :
-d "{ \"id\": 47, \"folderId\": 4, \"fileName\":\"Egzai_u002.doc\", \"templateId\":92241920, \"attributes\":[{\"name\":\"ack\",\"stringValue\":\"ack\",\"type\":0},{\"name\":\"Tar\",\"dateValue\":\"2017-03-18 19:10:00 +0100\",\"type\":3}] }" \
http://localhost:8080/services/rest/document/update
Creates a document with the create method on Windows 10<br/>
* YOLO
Windows 10, curl 7.55.1 (Windows) libcurl/7.55.1 WinSSL
* COCO
* Pascal VOC
* CSV
<pre>
For YOLO training , export the dataset in YOLO format .
curl -v -u admin:admin -X POST "http://localhost:8080/services/rest/document/create" -H "accept: application/json" -H "Content-Type: multipart/form-data" -F "document={ \"language\":\"en\",\"fileName\":\"ScreenHunter949.png\",\"folderId\":4 };type=application/json" -F "content=@C:\Users\shatz\Desktop\ScreenHunter949.png;type=application/octet-stream"
</pre>
Checkout an existing document<br/>
=== Dataset Formats ===
$ curl -k -u admin:admin -X POST https://[url]/services/rest/document/checkout \
-H "accept: application/json" -H "Content-Type: application/x-www-form-urlencoded" -d "docId=2686"
Checkin a new version of document (the document must be in checked-out state) [Windows 11]<br/>
==== COCO == ==
$ curl -v -u admin:admin -X POST "http://localhost:8080/services/rest/document/checkin" \
-H "accept: */*" -H "Content-Type: multipart/form-data" -F "docId=721" -F "comment=" -F "release=false" -F "filename=Checkmarx.txt" -F "filedata=@Checkmarx.txt;type=text/plain"
Upload: Creates a new document/Creates a new version of an existing document (it can do both things) [Windows 11]<br/>
COCO is a JSON-based dataset format commonly used for object detection datasets.
$ curl -u admin:admin -X POST "http://localhost:8080/services/rest/document/upload" \
More information :
-H "Accept: application/json" -H "Content-Type: multipart/form-data" -F "docId=721" -F "folderId=" -F "release=false" \
https ://docs.aws.amazon.com /rekognition /latest /customlabels -dg /md -coco -overview .html
-F "filename=logicaldoc_community - Checkmarx AST.pdf" -F "language=en" -F "filedata=@logicaldoc_community - Checkmarx AST.pdf;type=application/pdf"
== Search ==
==== YOLO ====
Standard Full-text search on content, title and tags using english as language of the query (expressionLanguage) and on english documents (language):
$ curl -u admin:admin -H "Content-Type: application/json" -H "Accept: application/json" -X POST \
YOLO datasets contain images and annotation files organized according to a predefined directory structure.
-d "{\"maxHits\":50,\"expression\":\"document management system\",\"expressionLanguage\":\"en\",\"language\":\"en\"}"
http://localhost:8080/services/rest/search/find
More info at:
More information :
https://docs.cvat.ai/docs/dataset_management/formats/format-yolo/
* [http://www.yilmazhuseyin.com/blog/dev/curl-tutorial-examples-usage/ curl tutorial with examples of usage]
==== YOLOv8 OBB ====
* [http://docs.logicaldoc.com/resources/wsdoc/rest/index.html?version=7.6.4 LogicalDOC REST API reference v7.6.4]
* [https://docs.logicaldoc.com/resources/wsdoc/rest/index.html?version=7.7.6 LogicalDOC REST API reference v7.7.6]
* [https://docs.logicaldoc.com/resources/wsdoc/rest/index.html?version=8.1.0 LogicalDOC REST API reference v8.1]
* [https://app.swaggerhub.com/apis/swatzniak/logicaldoc_rest_api/8.8.1 LogicalDOC REST API | 8.8.1 | SwaggerHub]
* [[Bindings_And_Samples|Webservices - Binding and Examples]]
* [https://docs.logicaldoc.com/en/web-services-api LogicalDOC Web Services API]
[[Category: RESTful Guide]]
YOLOv8 OBB (Oriented Bounding Boxes) extends the standard YOLO format by supporting rotated bounding boxes using eight normalized coordinates.
Preparing a Dataset with Label Studio This guide explains how to create an annotated dataset for YOLO training using Label Studio.
Install Label Studio Refer to the official installation guide:
https://labelstud.io/guide/install
Enable Local File Storage For large projects it is not recommended to upload images directly through the Label Studio interface.
To enable local file access, configure the following environment variables:
LABEL_STUDIO_LOCAL_FILES_SERVING_ENABLED=true
LABEL_STUDIO_LOCAL_FILES_DOCUMENT_ROOT=/path/to/images
Launch Label Studio:
label-studio start
Create a Project Login to Label Studio
Click Create Project
Enter a project name
Configure the labeling interface
Save the project Import Images Open the project
Click Import
Select Local Storage When importing images, choose Files as the import method.
Unlike CVAT, Label Studio creates one task for each imported document image.
File:LabelStudio-import-method.png Selecting the Files import method Annotate Documents Open a task
Select a label
Draw a bounding box around the target area
Save the annotation Example labels:
Invoice Number
Date
Seller Name
Buyer Name
Total Amount File:LabelStudio-annotation-example.png Example annotation Export the Dataset Open the project
Click Export
Select the desired format Supported formats include:
For YOLO training, export the dataset in YOLO format.
Dataset Formats COCO COCO is a JSON-based dataset format commonly used for object detection datasets.
More information:
https://docs.aws.amazon.com/rekognition/latest/customlabels-dg/md-coco-overview.html
YOLO YOLO datasets contain images and annotation files organized according to a predefined directory structure.
More information:
https://docs.cvat.ai/docs/dataset_management/formats/format-yolo/
YOLOv8 OBB YOLOv8 OBB (Oriented Bounding Boxes) extends the standard YOLO format by supporting rotated bounding boxes using eight normalized coordinates.
