wangtong/ISAT_with_sam
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

update

Browse Source
This commit is contained in:
yatengLG 2023-08-04 17:31:36 +08:00
parent 0f2d1e3d6b
commit 3bfe4ad17f
3 changed files with 154 additions and 42 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
README.md
View File

@ -78,7 +78,8 @@ python main.py
通过鼠标左键（或右键）提示感兴趣区域（或不感兴趣区域），调整目标分割掩码。
2.2 手动标注
点击工具栏[绘制多边形]开始手动标注（快捷键C）
通过鼠标左键添加多边形顶点，框取目标。
通过鼠标左键添加多边形顶点。
鼠标左键按下并拖动，持续添加顶点，时间间隔0.15s。
2.3 退上一个状态
工具栏点击工具栏[回退]按钮（快捷键Z），回退到标注的上一个状态。
半自动标注时，删除上一个添加的点提示；手动标注时，删除上一个添加的顶点。
@ -103,7 +104,7 @@ python main.py
点击工具栏[位图]按钮（快捷键SPACE），预览标注结果。
点击时，以 ‘标注-语义-实例’ 的顺序进行切换。
2. 窗口调整
点击工具栏[放大]（快捷键SPACE），[缩小]（快捷键SPACE），[适应窗口]（快捷键SPACE）调整图片大小。
点击工具栏[放大]，[缩小]，[适应窗口]（快捷键F）调整图片大小。
3. 显示/隐藏目标
点击工具栏[显示/隐藏]按钮（快捷键V），显示或隐藏当前已标注目标。
也可以在右侧标注栏中，通过勾选框显示/隐藏单个目标。

126
docs/README-en.md
View File

@ -3,9 +3,12 @@
![annotate.gif](./display/标注.gif)
Quick annotate for image segmentation by [segment anything](https://github.com/facebookresearch/segment-anything)
**If this project brings convenience to your work and life, please provide a Star; If you want to contribute code to this project, please send Pull requests**
**Project updating，suggestions welcome**
![](https://img.shields.io/github/stars/yatengLG/ISAT_with_segment_anything?style=social)
![](https://img.shields.io/github/forks/yatengLG/ISAT_with_segment_anything?style=social)
Quick annotate for image segmentation by [segment anything](https://github.com/facebookresearch/segment-anything)
Demo Video：[youtube](https://www.youtube.com/watch?v=yLdZCPmX-Bc)
@ -15,66 +18,113 @@ Demo Video：[youtube](https://www.youtube.com/watch?v=yLdZCPmX-Bc)
# Feature
1. Support semantic segmentation and instance segmentation.
2. Integrating SAM (segment anything model) for interactive semi-automatic annotation of image segmentation.
3. Interactive correction mask, by clicking on the area of interest (not of interest) with the left (right) mouse button.
4. Support manual creation of polygons.
5. Support modifying polygons.
6. Support for adjusting polygon occlusion.
7. Support preview annotation result.
8. ISAT format json, contains more information.
9. Support opening JSON files annotated by labelme (please backup a copy before opening), and modify.
10. Support exporting ISAT format json to VOC as single-channel png images.
11. Support exporting ISAT format jsons to COCO format json.
12. Support exporting ISAT format json to LabelMe format json.
13. Support exporting COCO format json to ISAT format jsons.
- Support interactive semi-automatic annotation based on SAM.
- Support manual creation of polygons.
- Support modifying polygons.
- Support for adjusting polygon occlusion.
- Support preview annotation result.
- More features refer to [Features Description](features%20description.md)
# INSTALL
## 1. Run the source code
## 1. Run with source code
### (1) Create environment
```shell
conda create -n ISAT_with_segment_anything python==3.8
conda activate ISAT_with_segment_anything
```
### (2) Install Segment anything
```shell
git clone https://github.com/facebookresearch/segment-anything.git
cd segment-anything
pip install -e .
cd ..
```
### (3) Install ISAT_with_segment_anything
### (2) Install ISAT_with_segment_anything
```shell
git clone https://github.com/yatengLG/ISAT_with_segment_anything.git
cd ISAT_with_segment_anything
pip install -r requirements.txt
```
### (4) Download Segment anything pretrained checkpoint.
### (3) Download Segment anything pretrained checkpoint.
Download the checkpoint，and save in the path: ISAT_with_segment_anything/segment_any
- H-checkpoint:[sam_vit_h_4b8939.pth](https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth)
H checkpoint has best effect, but need more resources.VRAM needs at least 8G.
- L-checkpoint:[sam_vit_l_0b3195.pth](https://dl.fbaipublicfiles.com/segment_anything/sam_vit_l_0b3195.pth)
Now support [SAM](https://github.com/facebookresearch/segment-anything)，[sam-hq](https://github.com/SysCV/sam-hq)，[MobileSAM](https://github.com/ChaoningZhang/MobileSAM)。
L checkpoint has normal effect and normal resources.VRAM needs at least 7G.
- B-checkpoint:[sam_vit_b_01ec64.pth](https://dl.fbaipublicfiles.com/segment_anything/sam_vit_b_01ec64.pth)
B checkpoint has pool effect, but need less resources.VRAM needs at least 6G.
| | pretrained checkpoint | memory | size |
|----|----|----|----|
| SAM | [sam_vit_h_4b8939.pth](https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth) | 7305M | 2.6G |
| | [sam_vit_l_0b3195.pth](https://dl.fbaipublicfiles.com/segment_anything/sam_vit_l_0b3195.pth) | 5855M | 2.6G |
| | [sam_vit_b_01ec64.pth](https://dl.fbaipublicfiles.com/segment_anything/sam_vit_b_01ec64.pth) | 4149M | 375M |
| sam-hq | [sam_hq_vit_h.pth](https://huggingface.co/lkeab/hq-sam/blob/main/sam_hq_vit_h.pth) | 7393M | 2.6G |
| | [sam_hq_vit_l.pth](https://huggingface.co/lkeab/hq-sam/blob/main/sam_hq_vit_l.pth) | 5939M | 1.3G |
| | [sam_hq_vit_b.pth](https://huggingface.co/lkeab/hq-sam/blob/main/sam_hq_vit_b.pth) | 4207M | 379M |
| | [sam_hq_vit_tiny.pth](https://huggingface.co/lkeab/hq-sam/blob/main/sam_hq_vit_tiny.pth) | 1463M | 43M |
| mobile-sam | [mobile_sam.pt](https://github.com/ChaoningZhang/MobileSAM/blob/master/weights/mobile_sam.pt)| 1375M | 40M |
### (5) Run
```shell
python main.py
```
## 2. run exe
### (1) download exe
**Coming soon**
### (2) Download Segment anything pretrained checkpoint.
# Explain
1. The software needs to run well under the GPU and enought VRAM.
2. If you don't have a GPU or don't have enought VARM, please manually draw polygons for labeling by [ISAT](https://github.com/yatengLG/ISAT).
3.
# Use
## 1.Annotate
```text
1. Choice the category in left window of software.
Edit category in Toolbar-File-Setting.
2. Start annotate
2.1 semi-automatic annotate with SAM.
Click button named [Segment anything] start annotate(shortcut Q).
Click interested area with left button of mouse, click uninterested area with right button of mouse, SAM will calcute masks.
2.2 manual creation of polygons.
Click button named [Draw polygon] start annotate(shortcut C).
Click with left button of mouse add point into the polygon.
Press left button of mouse and drag will auto add point into the polygon, time interval of 0.15 seconds
2.3 backspace
Click button named [Backspace] return previous state
(shortcut Z).
3. Click button named [Annotate finished] finish annotate.
(shortcut E).
4. Click button named [Save] write annotations to json file.
(shortcut S).
```
## 2.Modify
```text
1. Polygon modify
Drag and drop polygon vertices to modify the shape of the polygon.
Drag the polygon to adjust its position.
2. Category modify
Chioce the polygon and click button named [Edit] or double click polygon, choice new category in edit window.
3. Occlusion modify
Chioce the polygon and click button named [To top](shortcut T) or [To bottom](shortcut B).
4. Delete polygon
Chioce the polygon and click button named [Delete] to delete the polygon.
```
## 3.View
```text
1. Preview annotation
Click button named [Bit map], preview semantic and instance annotation result.(shortcut space)
The order of swithing is polygons-semantic-instance.
2. Image window
Click Buttons named [Zoom in],[Zoom out],[Fit window](shortcut F)Adjust the size of image.
3. show/hide polygons
Click button named [Visible] ,show/hide polygons.(shortcut V)
4. mask aplha(only effective when using SAM)
Drag mask aplha bar to adjust the mask contrast ratio.
```
## 4.Data convert
ISAT have specific format with json.You can use convert tools or convert to other format by yourself.
```text
1. ISAT to VOC
Convert ISAT jsons to PNG images.
2. ISAT to COCO
Convert ISAT jsons to COCO json.
3. ISAT to LABELME
Convert ISAT jsons to LABELME jsons.
4. COCO to ISAT
Convert COCO json to ISAT jsons.
```
# Citation
```text
@ -82,7 +132,7 @@ python main.py
title={{ISAT with segment anything}: Image segmentation annotation tool with segment anything},
url={https://github.com/yatengLG/ISAT_with_segment_anything},
note={Open source software available from https://github.com/yatengLG/ISAT_with_segment_anything},
author={yatengLG and horffmanwang},
author={yatengLG, Alias-z and horffmanwang},
year={2023},
}
```

61
docs/features description.md Normal file
View File

@ -0,0 +1,61 @@
- Language switching
- Model switching
- Contour mode
- Occlusion adjustment
- Quick jump to image
- Statusbar information
- Preview annotation result
- Import/export config file
# 1.Language switching
The software provides two interfaces, Chinese and English, which can be switched at any time.
![双语界面.gif](../display/双语界面.gif)
# 2.Model switching
Switching model in your download models.
![模型切换.gif](../display/模型切换.gif)
# 3.Contour mode
When using SAM for semi-automatic annotation,convert SAM mask to contours by opencv, and then convert contours to polygons.
1. Max only.
```text
Usually,the contour with the highest number of vertices also has the largest area.
Other contours will delete as noise.
```
2. External.
```text
Save external contours, the hole of masks will be filled.
```
3. All.
```text
Save all contours, will add polygons with category __background__ for holes.
```
![轮廓保存模式.gif](../display/轮廓保存模式.gif)
# 4.Occlusion adjustment
For polygons with overlapping parts,adjustment occlusion with **to top** or **to bottom**.
![图层调整遮挡关系.gif](../display/图层调整遮挡关系.gif)
# 5.Quick jump to image
Input image name or index to jump the image.
![图片快速跳转.gif](../display/图片快速跳转.gif)
# 6.Statusbar information
Get position and value of pixel.
If use SAM also show cuda memory in statusbar.
![状态栏信息.gif](../display/状态栏信息.gif)
# 7.Preview annotation result
Click **Bit map** button to preview semantic and instance annotation result.
![实时预览.gif](../display/实时预览.gif)
# 8.Import/export config file
Use config file save categories and other args.
![配置文件导入导出.gif](../display/配置文件导入导出.gif)