{ "cells": [ { "cell_type": "markdown", "id": "c702f5b3-388a-45d9-bb5d-fde8f3b0f21c", "metadata": {}, "source": [ "# Morphological operations on label images\n", "\n", "In this notebook we will demonstrate how to fine-tune outlines of labels by smoothing them. The operation is related to erosion and dilation of labels. It is however not exactly what opening does. It differs by not leaving gaps behind but filling them with the closes labels.\n", "\n", "See also\n", "* [Twitter discussion about naming 'smooth_labels'](https://twitter.com/haesleinhuepf/status/1492215964305436673)" ] }, { "cell_type": "code", "execution_count": 5, "id": "de9bd90b-74ee-479f-ab76-badd2479375a", "metadata": {}, "outputs": [], "source": [ "import numpy as np\n", "import pyclesperanto_prototype as cle\n", "import napari_segment_blobs_and_things_with_membranes as nsbatwm\n", "import matplotlib.pyplot as plt\n", "from skimage.data import cells3d\n", "import stackview" ] }, { "cell_type": "markdown", "id": "252b3f9c-2184-496a-8640-9d0021d7cc00", "metadata": {}, "source": [ "A potential use-case is fine-tuning cell segmentation results. Thus, we take a look at a segmentation of cells based on membranes." ] }, { "cell_type": "code", "execution_count": 4, "id": "20e78e5e-bb1d-4b1f-a065-8a45315c86e9", "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "
\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
shape(256, 256)
dtypeuint16
size128.0 kB
min277
max44092
\n", "\n", "
" ], "text/plain": [ "StackViewNDArray([[4496, 5212, 6863, ..., 2917, 2680, 2642],\n", " [4533, 5146, 7555, ..., 2843, 2857, 2748],\n", " [4640, 6082, 8452, ..., 3372, 3039, 3128],\n", " ...,\n", " [1339, 1403, 1359, ..., 4458, 4314, 4795],\n", " [1473, 1560, 1622, ..., 3967, 4531, 4204],\n", " [1380, 1368, 1649, ..., 3091, 3558, 3682]], dtype=uint16)" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "membranes = cells3d()[30,0]\n", "stackview.insight(membranes)" ] }, { "cell_type": "code", "execution_count": 6, "id": "450218df-5118-4d03-bfc7-585517639ada", "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "
\n", "\n", "\n", "nsbatwm made image
\n", "\n", "\n", "\n", "\n", "\n", "
shape(256, 256)
dtypeint32
size256.0 kB
min1
max27
\n", "\n", "
" ], "text/plain": [ "StackViewNDArray([[ 5, 5, 5, ..., 3, 3, 3],\n", " [ 5, 5, 5, ..., 3, 3, 3],\n", " [ 5, 5, 5, ..., 3, 3, 3],\n", " ...,\n", " [24, 24, 24, ..., 27, 27, 27],\n", " [24, 24, 24, ..., 27, 27, 27],\n", " [24, 24, 24, ..., 27, 27, 27]])" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "labels = nsbatwm.local_minima_seeded_watershed(membranes)\n", "labels" ] }, { "cell_type": "markdown", "id": "a41aa9d2-b2d9-4f14-aa1e-1203a1a2a7ab", "metadata": {}, "source": [ "## Label erosion\n", "Label erosion works exactly like erosion on binary images. The only difference is that it works on a per-label basis. Each label is eroded independently from the others introducing black (`0`) regions between them." ] }, { "cell_type": "code", "execution_count": 8, "id": "1f45f51e-773e-4b6b-8511-228e7e6c8029", "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "
\n", "\n", "\n", "cle._ image
\n", "\n", "\n", "\n", "\n", "\n", "
shape(256, 256)
dtypeuint32
size256.0 kB
min0.0
max27.0
\n", "\n", "
" ], "text/plain": [ "cl.OCLArray([[ 5, 5, 5, ..., 3, 3, 3],\n", " [ 5, 5, 5, ..., 0, 0, 0],\n", " [ 5, 5, 5, ..., 0, 0, 0],\n", " ...,\n", " [24, 24, 24, ..., 0, 0, 0],\n", " [24, 24, 24, ..., 0, 0, 0],\n", " [24, 24, 24, ..., 0, 0, 0]], dtype=uint32)" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "eroded_labels = cle.erode_labels(labels, radius=5)\n", "\n", "eroded_labels" ] }, { "cell_type": "markdown", "id": "0b90e469-de37-4bcb-b1a1-0a8d8c18de33", "metadata": {}, "source": [ "## Label dilation\n", "To close these black gaps again, we can apply label dilation. It is different from binary image dilation as labels cannot overwrite each other. When the gap between them is closed, the labels stop growing." ] }, { "cell_type": "code", "execution_count": 10, "id": "efba5780-bda3-4286-864f-1fd56afa37ec", "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "
\n", "\n", "\n", "cle._ image
\n", "\n", "\n", "\n", "\n", "\n", "
shape(256, 256)
dtypeuint32
size256.0 kB
min0.0
max27.0
\n", "\n", "
" ], "text/plain": [ "cl.OCLArray([[ 5, 5, 5, ..., 3, 3, 3],\n", " [ 5, 5, 5, ..., 3, 3, 3],\n", " [ 5, 5, 5, ..., 3, 3, 3],\n", " ...,\n", " [24, 24, 24, ..., 0, 0, 0],\n", " [24, 24, 24, ..., 0, 0, 0],\n", " [24, 24, 24, ..., 0, 0, 0]], dtype=uint32)" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "dilated_labels = cle.dilate_labels(eroded_labels, radius=5)\n", "dilated_labels" ] }, { "cell_type": "markdown", "id": "a2e69606-95c1-4da2-8e8b-5c8334b964d6", "metadata": {}, "source": [ "## Label opening and closing\n", "Combining label erosion and dilation as shown above can also be done in one shot using label closing and opening." ] }, { "cell_type": "code", "execution_count": 11, "id": "ce4d3c0a-0fde-4b93-b4bf-98e81387a0cb", "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "
\n", "\n", "\n", "cle._ image
\n", "\n", "\n", "\n", "\n", "\n", "
shape(256, 256)
dtypeuint32
size256.0 kB
min0.0
max27.0
\n", "\n", "
" ], "text/plain": [ "cl.OCLArray([[ 5, 5, 5, ..., 3, 3, 3],\n", " [ 5, 5, 5, ..., 3, 3, 3],\n", " [ 5, 5, 5, ..., 3, 3, 3],\n", " ...,\n", " [24, 24, 24, ..., 0, 0, 0],\n", " [24, 24, 24, ..., 0, 0, 0],\n", " [24, 24, 24, ..., 0, 0, 0]], dtype=uint32)" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "opened_labels = cle.opening_labels(labels, radius=5)\n", "opened_labels" ] }, { "cell_type": "markdown", "id": "59465a6b-6e51-4706-8d25-713dcd15da5e", "metadata": {}, "source": [ "## Smoothing labels\n", "An operation similar to label opening is label smoothing: It just prevents that black gaps are introduced. The `smooth_labels` function allows to straighten the outlines of the labels." ] }, { "cell_type": "code", "execution_count": 12, "id": "c5312691-1ed3-4e2c-8037-7425d7df0c45", "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "
\n", "\n", "\n", "cle._ image
\n", "\n", "\n", "\n", "\n", "\n", "
shape(256, 256)
dtypeuint32
size256.0 kB
min1.0
max27.0
\n", "\n", "
" ], "text/plain": [ "cl.OCLArray([[ 5, 5, 5, ..., 3, 3, 3],\n", " [ 5, 5, 5, ..., 3, 3, 3],\n", " [ 5, 5, 5, ..., 3, 3, 3],\n", " ...,\n", " [24, 24, 24, ..., 22, 22, 22],\n", " [24, 24, 24, ..., 27, 22, 22],\n", " [24, 24, 24, ..., 27, 27, 22]], dtype=uint32)" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "cle.smooth_labels(labels, radius=5)" ] }, { "cell_type": "markdown", "id": "d5c8787a-63cc-4989-bcf0-59fd197f80eb", "metadata": {}, "source": [ "## Exercise\n", "Use `stackview.interact` to play with the `radius` parameter of `cle.opening_labels` and `cle.smooth_labels`. In case there is an error message, make sure that `stackview>=0.6.2` installed." ] }, { "cell_type": "code", "execution_count": 14, "id": "213865ff-5451-4cff-94de-1618bdb6e4cb", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'0.6.1'" ] }, "execution_count": 14, "metadata": {}, "output_type": "execute_result" } ], "source": [ "stackview.__version__" ] }, { "cell_type": "code", "execution_count": 15, "id": "d1444682-6faf-4820-95bc-47d8e052e4a0", "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "37382fea14f745059c863397035fc91f", "version_major": 2, "version_minor": 0 }, "text/plain": [ "VBox(children=(interactive(children=(IntSlider(value=0, description='radius'), Output()), _dom_classes=('widge…" ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "stackview.interact(cle.opening_labels, labels)" ] }, { "cell_type": "code", "execution_count": 13, "id": "aa673421-1930-4f28-b285-21c561dc182f", "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "86faae3c6ca246f69a8a744414b2076b", "version_major": 2, "version_minor": 0 }, "text/plain": [ "VBox(children=(interactive(children=(IntSlider(value=0, description='radius'), Output()), _dom_classes=('widge…" ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "stackview.interact(cle.smooth_labels, labels)" ] }, { "cell_type": "markdown", "id": "cc2a6cc7-8234-4e97-bcd7-2d972f23b96c", "metadata": {}, "source": [ "## Exercise\n", "Apply a maximum filter to the `eroded_labels` image. Try different radii. Find out why applying this function is not a good idea in practice." ] }, { "cell_type": "code", "execution_count": null, "id": "f31d2788-3aaa-4454-b877-5c4cf4d71648", "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.15" } }, "nbformat": 4, "nbformat_minor": 5 }