From 3bfe4ad17f93a6ec940d765eb3f31838fc12c487 Mon Sep 17 00:00:00 2001 From: yatengLG <767624851@qq.com> Date: Fri, 4 Aug 2023 17:31:36 +0800 Subject: [PATCH] update --- README.md | 5 +- docs/README-en.md | 130 ++++++++++++++++++++++++----------- docs/features description.md | 61 ++++++++++++++++ 3 files changed, 154 insertions(+), 42 deletions(-) create mode 100644 docs/features description.md diff --git a/README.md b/README.md index d05ee6f..158d6f6 100644 --- a/README.md +++ b/README.md @@ -78,7 +78,8 @@ python main.py 通过鼠标左键（或右键）提示感兴趣区域（或不感兴趣区域），调整目标分割掩码。 2.2 手动标注 点击工具栏[绘制多边形]开始手动标注（快捷键C） - 通过鼠标左键添加多边形顶点，框取目标。 + 通过鼠标左键添加多边形顶点。 + 鼠标左键按下并拖动，持续添加顶点，时间间隔0.15s。 2.3 退上一个状态 工具栏点击工具栏[回退]按钮（快捷键Z），回退到标注的上一个状态。 半自动标注时，删除上一个添加的点提示；手动标注时，删除上一个添加的顶点。 @@ -103,7 +104,7 @@ python main.py 点击工具栏[位图]按钮（快捷键SPACE），预览标注结果。 点击时，以 ‘标注-语义-实例’ 的顺序进行切换。 2. 窗口调整 - 点击工具栏[放大]（快捷键SPACE），[缩小]（快捷键SPACE），[适应窗口]（快捷键SPACE）调整图片大小。 + 点击工具栏[放大]，[缩小]，[适应窗口]（快捷键F）调整图片大小。 3. 显示/隐藏目标 点击工具栏[显示/隐藏]按钮（快捷键V），显示或隐藏当前已标注目标。 也可以在右侧标注栏中，通过勾选框显示/隐藏单个目标。 diff --git a/docs/README-en.md b/docs/README-en.md index d4c2a08..7d5ca8b 100644 --- a/docs/README-en.md +++ b/docs/README-en.md @@ -3,9 +3,12 @@ ![annotate.gif](./display/标注.gif) -Quick annotate for image segmentation by [segment anything](https://github.com/facebookresearch/segment-anything) +**If this project brings convenience to your work and life, please provide a Star; If you want to contribute code to this project, please send Pull requests** -**Project updating，suggestions welcome** +![](https://img.shields.io/github/stars/yatengLG/ISAT_with_segment_anything?style=social) +![](https://img.shields.io/github/forks/yatengLG/ISAT_with_segment_anything?style=social) + +Quick annotate for image segmentation by [segment anything](https://github.com/facebookresearch/segment-anything) Demo Video：[youtube](https://www.youtube.com/watch?v=yLdZCPmX-Bc) @@ -15,66 +18,113 @@ Demo Video：[youtube](https://www.youtube.com/watch?v=yLdZCPmX-Bc) # Feature -1. Support semantic segmentation and instance segmentation. -2. Integrating SAM (segment anything model) for interactive semi-automatic annotation of image segmentation. -3. Interactive correction mask, by clicking on the area of interest (not of interest) with the left (right) mouse button. -4. Support manual creation of polygons. -5. Support modifying polygons. -6. Support for adjusting polygon occlusion. -7. Support preview annotation result. -8. ISAT format json, contains more information. -9. Support opening JSON files annotated by labelme (please backup a copy before opening), and modify. -10. Support exporting ISAT format json to VOC as single-channel png images. -11. Support exporting ISAT format jsons to COCO format json. -12. Support exporting ISAT format json to LabelMe format json. -13. Support exporting COCO format json to ISAT format jsons. +- Support interactive semi-automatic annotation based on SAM. +- Support manual creation of polygons. +- Support modifying polygons. +- Support for adjusting polygon occlusion. +- Support preview annotation result. +- More features refer to [Features Description](features%20description.md) # INSTALL -## 1. Run the source code +## 1. Run with source code ### (1) Create environment ```shell conda create -n ISAT_with_segment_anything python==3.8 conda activate ISAT_with_segment_anything ``` -### (2) Install Segment anything -```shell -git clone https://github.com/facebookresearch/segment-anything.git -cd segment-anything -pip install -e . -cd .. -``` - -### (3) Install ISAT_with_segment_anything +### (2) Install ISAT_with_segment_anything ```shell git clone https://github.com/yatengLG/ISAT_with_segment_anything.git cd ISAT_with_segment_anything pip install -r requirements.txt ``` -### (4) Download Segment anything pretrained checkpoint. +### (3) Download Segment anything pretrained checkpoint. Download the checkpoint，and save in the path: ISAT_with_segment_anything/segment_any -- H-checkpoint:[sam_vit_h_4b8939.pth](https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth) - - H checkpoint has best effect, but need more resources.VRAM needs at least 8G. -- L-checkpoint:[sam_vit_l_0b3195.pth](https://dl.fbaipublicfiles.com/segment_anything/sam_vit_l_0b3195.pth) - - L checkpoint has normal effect and normal resources.VRAM needs at least 7G. -- B-checkpoint:[sam_vit_b_01ec64.pth](https://dl.fbaipublicfiles.com/segment_anything/sam_vit_b_01ec64.pth) - - B checkpoint has pool effect, but need less resources.VRAM needs at least 6G. + +Now support [SAM](https://github.com/facebookresearch/segment-anything)，[sam-hq](https://github.com/SysCV/sam-hq)，[MobileSAM](https://github.com/ChaoningZhang/MobileSAM)。 + +| | pretrained checkpoint | memory | size | +|----|----|----|----| +| SAM | [sam_vit_h_4b8939.pth](https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth) | 7305M | 2.6G | +| | [sam_vit_l_0b3195.pth](https://dl.fbaipublicfiles.com/segment_anything/sam_vit_l_0b3195.pth) | 5855M | 2.6G | +| | [sam_vit_b_01ec64.pth](https://dl.fbaipublicfiles.com/segment_anything/sam_vit_b_01ec64.pth) | 4149M | 375M | +| sam-hq | [sam_hq_vit_h.pth](https://huggingface.co/lkeab/hq-sam/blob/main/sam_hq_vit_h.pth) | 7393M | 2.6G | +| | [sam_hq_vit_l.pth](https://huggingface.co/lkeab/hq-sam/blob/main/sam_hq_vit_l.pth) | 5939M | 1.3G | +| | [sam_hq_vit_b.pth](https://huggingface.co/lkeab/hq-sam/blob/main/sam_hq_vit_b.pth) | 4207M | 379M | +| | [sam_hq_vit_tiny.pth](https://huggingface.co/lkeab/hq-sam/blob/main/sam_hq_vit_tiny.pth) | 1463M | 43M | +| mobile-sam | [mobile_sam.pt](https://github.com/ChaoningZhang/MobileSAM/blob/master/weights/mobile_sam.pt)| 1375M | 40M | ### (5) Run ```shell python main.py ``` +## 2. run exe +### (1) download exe +**Coming soon** +### (2) Download Segment anything pretrained checkpoint. -# Explain -1. The software needs to run well under the GPU and enought VRAM. -2. If you don't have a GPU or don't have enought VARM, please manually draw polygons for labeling by [ISAT](https://github.com/yatengLG/ISAT). -3. +# Use +## 1.Annotate +```text +1. Choice the category in left window of software. + Edit category in Toolbar-File-Setting. + +2. Start annotate + 2.1 semi-automatic annotate with SAM. + Click button named [Segment anything] start annotate(shortcut Q). + Click interested area with left button of mouse, click uninterested area with right button of mouse, SAM will calcute masks. + 2.2 manual creation of polygons. + Click button named [Draw polygon] start annotate(shortcut C). + Click with left button of mouse add point into the polygon. + Press left button of mouse and drag will auto add point into the polygon, time interval of 0.15 seconds + 2.3 backspace + Click button named [Backspace] return previous state +(shortcut Z). +3. Click button named [Annotate finished] finish annotate. +(shortcut E). +4. Click button named [Save] write annotations to json file. +(shortcut S). +``` +## 2.Modify +```text +1. Polygon modify + Drag and drop polygon vertices to modify the shape of the polygon. + Drag the polygon to adjust its position. +2. Category modify + Chioce the polygon and click button named [Edit] or double click polygon, choice new category in edit window. +3. Occlusion modify + Chioce the polygon and click button named [To top](shortcut T) or [To bottom](shortcut B). +4. Delete polygon + Chioce the polygon and click button named [Delete] to delete the polygon. +``` +## 3.View +```text +1. Preview annotation + Click button named [Bit map], preview semantic and instance annotation result.(shortcut space) + The order of swithing is polygons-semantic-instance. +2. Image window + Click Buttons named [Zoom in],[Zoom out],[Fit window](shortcut F)Adjust the size of image. +3. show/hide polygons + Click button named [Visible] ,show/hide polygons.(shortcut V) +4. mask aplha(only effective when using SAM) + Drag mask aplha bar to adjust the mask contrast ratio. +``` +## 4.Data convert +ISAT have specific format with json.You can use convert tools or convert to other format by yourself. +```text +1. ISAT to VOC + Convert ISAT jsons to PNG images. +2. ISAT to COCO + Convert ISAT jsons to COCO json. +3. ISAT to LABELME + Convert ISAT jsons to LABELME jsons. +4. COCO to ISAT + Convert COCO json to ISAT jsons. +``` # Citation ```text @@ -82,7 +132,7 @@ python main.py title={{ISAT with segment anything}: Image segmentation annotation tool with segment anything}, url={https://github.com/yatengLG/ISAT_with_segment_anything}, note={Open source software available from https://github.com/yatengLG/ISAT_with_segment_anything}, - author={yatengLG and horffmanwang}, + author={yatengLG, Alias-z and horffmanwang}, year={2023}, } ``` \ No newline at end of file diff --git a/docs/features description.md b/docs/features description.md new file mode 100644 index 0000000..811b93d --- /dev/null +++ b/docs/features description.md @@ -0,0 +1,61 @@ +- Language switching +- Model switching +- Contour mode +- Occlusion adjustment +- Quick jump to image +- Statusbar information +- Preview annotation result +- Import/export config file + +# 1.Language switching +The software provides two interfaces, Chinese and English, which can be switched at any time. + +![双语界面.gif](../display/双语界面.gif) + +# 2.Model switching +Switching model in your download models. + +![模型切换.gif](../display/模型切换.gif) + +# 3.Contour mode +When using SAM for semi-automatic annotation,convert SAM mask to contours by opencv, and then convert contours to polygons. + +1. Max only. +```text +Usually,the contour with the highest number of vertices also has the largest area. +Other contours will delete as noise. +``` +2. External. +```text +Save external contours, the hole of masks will be filled. +``` +3. All. +```text +Save all contours, will add polygons with category __background__ for holes. +``` +![轮廓保存模式.gif](../display/轮廓保存模式.gif) + + +# 4.Occlusion adjustment +For polygons with overlapping parts,adjustment occlusion with **to top** or **to bottom**. + +![图层调整遮挡关系.gif](../display/图层调整遮挡关系.gif) + + +# 5.Quick jump to image +Input image name or index to jump the image. + +![图片快速跳转.gif](../display/图片快速跳转.gif) + +# 6.Statusbar information +Get position and value of pixel. +If use SAM also show cuda memory in statusbar. +![状态栏信息.gif](../display/状态栏信息.gif) + +# 7.Preview annotation result +Click **Bit map** button to preview semantic and instance annotation result. +![实时预览.gif](../display/实时预览.gif) + +# 8.Import/export config file +Use config file save categories and other args. +![配置文件导入导出.gif](../display/配置文件导入导出.gif)