<p>You didn't show us how the original .h5ad object was generated, how you read it in, how you wrote it, and if you did anything else in-between besides filtering. <a href="https://anndata.readthedocs.io/en/latest/generated/anndata.AnnData.write.html" rel="nofollow noreferrer"><code>AnnData.write()</code></a> takes a <code>compression</code> arg that might explain the difference. As also mentioned in that link, ensuring you are writing a sparse matrix can make a substantial difference to file size (the matrix in <code>adata.X</code> is generally by far the largest part of the object)</p>

CC BY-SA 4.0
2023-05-20T06:28:02.827Z
I was taking this dataset https://data.humancellatlas.org/explore/projects/85a9263b-0887-48ed-ab1a-ddfa773727b6/get-curl-command
PesKchan
CC BY-SA 4.0
2023-05-20T06:32:12.760Z
I have updated the question
PesKchan
CC BY-SA 4.0
2023-05-22T20:58:16.173Z
I have updated my code please have a look and for this "You didn't show us how the original .h5ad object was generated" please have a look https://data.humancellatlas.org/explore/projects/85a9263b-0887-48ed-ab1a-ddfa773727b6
PesKchan
CC BY-SA 4.0
2023-05-19T20:20:01.800Z
2023-05-19T20:20:01.800Z
Chris_Rands
<p>Scanpy's <code>h5ad</code> is in fact hdf5 format. This format is used for rapid read/write (i/o) large amounts of data and thats its purpose. Its fairly easy to use from scratch.</p>
<p>Its purpose is not to compress data. If you are concerned with the contents of <code>h5ad</code> why not just read them directly via the <code>hdf5</code> viewer. You can download it here. <a href="https://www.hdfgroup.org/downloads/hdfview/" rel="nofollow noreferrer">https://www.hdfgroup.org/downloads/hdfview/</a></p>
<p>The file will then open up as a &quot;Explorer&quot; (Windows) or &quot;Finder&quot; (OSX) GUI file system and you can open the various files and see whats in them.</p>
<p>My suspicion is that the original data is been written into the file. 'cause it can i/o rapidly and access precisely what it needs then file size could be a secondary consideration. Anyway you can peer in and find out by looking at the original file and the hdf5 save you've done.</p>
<p>An example from the website is here</p>
<p><a href="https://i.sstatic.net/zhtSf.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/zhtSf.png" alt="enter image description here" /></a></p>
<p>Cool right?</p>
<p>Personally, I think HDF5 is overrated and <code>feather</code> outperforms it for speed (i/o) - but thats just for dataframes. Its not much faster than pickling in case you are interested and pickling in addition does compression. parquet is the format that does rapid i/o but specialises in compression, again mostly for dataframes.</p>
<p>HDF5 however does permit very pretty GUI interfaces.</p>
<hr />
<p>Also if you want to load up and look at various files, that just uses the Python <code>hdf5</code> module.</p>

CC BY-SA 4.0
2023-05-18T02:33:53.890Z
2023-05-18T03:09:54.613Z
2023-05-18T03:09:54.613Z
M__
<p>I'm trying to do simple filter for single data that is stored in <code>h5ad</code> file format using this</p>
<pre><code>import scanpy as sc

sc.pp.filter_cells(adata, min_genes=500)
sc.pp.filter_genes(adata, min_cells=10)
print(adata.n_obs, adata.n_vars)
</code></pre>
<p>when I saved the filtered data and see the size of my original data compare it, the size of the filtered data is more, but the dimension etc are reduced due to filtering.</p>
<p>So is it something wrong am I doing?</p>
<p><strong>UPDATE</strong></p>
<pre><code>adata = sc.read_h5ad('/Single_cell/cerebellar_development/508cf892-174c-45ab-b2dc-05f54f1ee7ed/aldinger20.processed.h5ad')
adata

AnnData object with n_obs × n_vars = 69174 × 49166
    obs: 'orig.ident', 'nCount_RNA', 'nFeature_RNA', 'sample_id', 'percent.mito', 'S.Score', 'G2M.Score', 'Phase', 'CC.Difference', 'nCount_SCT', 'nFeature_SCT', 'age', 'figure_clusters', 'sex', 'type', 'experiment', 'fig_cell_type'
    var: 'name'
    obsm: 'X_pca', 'X_umap', 'X_tsne'


sc.pp.filter_cells(adata, min_genes=500)
sc.pp.filter_genes(adata, min_cells=10)
print(adata.n_obs, adata.n_vars)

results_file = 'adata.h5ad'
adata.write(results_file)
</code></pre>
<p><strong>FULL CODE</strong></p>
<h1>updated version: h5ad filter, added layers to make the filtered file usable for seurat to read and make a correct seurat object</h1>
<pre><code>#https://github.com/satijalab/seurat/issues/2485

import scanpy as sc
import numpy as np
import pandas as pd
import time
from ipywidgets import interact, interactive, widgets
from IPython.display import display, HTML

def filter_and_save_h5ad(h5ad_file, min_genes, min_cells):
    # h5ad file read
    adata = sc.read_h5ad(h5ad_file)

    # cells and genes filter parameter from user
    sc.pp.filter_cells(adata, min_genes=min_genes)
    sc.pp.filter_genes(adata, min_cells=min_cells)

    # Add 'counts' layer
    adata.layers['counts'] = adata.X.copy()

    # add 'winsorization' layer
    adata.layers['winsorized'] = adata.X.copy()


    # filtered dataset save
    filtered_file = h5ad_file.replace('.h5ad', '_filtered.h5ad')
    adata.write(filtered_file)

    print(f&quot;Filtered dataset saved to {filtered_file}&quot;)

# input widgets
h5ad_file_widget = widgets.Text(description='File_add:')
min_genes_widget = widgets.IntText(description='Min_Genes:')
min_cells_widget = widgets.IntText(description='Min_Cells:')
button = widgets.Button(description='Filter_Save')

# timer widget
time_widget = widgets.HTML(value=&quot;&lt;p&gt;Time passed away: 0 seconds&lt;/p&gt;&quot;)

def on_button_clicked(click_me):
    h5ad_file = h5ad_file_widget.value.strip()
    min_genes = min_genes_widget.value
    min_cells = min_cells_widget.value
    
    # timer
    start_time = time.time()
    
    filter_and_save_h5ad(h5ad_file, min_genes, min_cells)
    
    # Calculate and update the elapsed time
    elapsed_time = time.time() - start_time
    time_widget.value = f&quot;&lt;p&gt;Time passed away: {elapsed_time} seconds&lt;/p&gt;&quot;

button.on_click(on_button_clicked)

# Display the widgets and button
display(h5ad_file_widget, min_genes_widget, min_cells_widget, button, time_widget)
</code></pre>

<scrnaseq><single-cell>
h5ad file format filter
# h5ad file format filter

(Asked by: username_0 on 2023-05-17)

I'm trying to do simple filter for single data that is stored in h5ad file format using this
import scanpy as sc

sc.pp.filter_cells(adata, min_genes=500)
sc.pp.filter_genes(adata, min_cells=10)
print(adata.n_obs, adata.n_vars)

when I saved the filtered data and see the size of my original data compare it, the size of the filtered data is more, but the dimension etc are reduced due to filtering.
So is it something wrong am I doing?
UPDATE
adata = sc.read_h5ad('/Single_cell/cerebellar_development/508cf892-174c-45ab-b2dc-05f54f1ee7ed/aldinger20.processed.h5ad')
adata

AnnData object with n_obs × n_vars = 69174 × 49166
    obs: 'orig.ident', 'nCount_RNA', 'nFeature_RNA', 'sample_id', 'percent.mito', 'S.Score', 'G2M.Score', 'Phase', 'CC.Difference', 'nCount_SCT', 'nFeature_SCT', 'age', 'figure_clusters', 'sex', 'type', 'experiment', 'fig_cell_type'
    var: 'name'
    obsm: 'X_pca', 'X_umap', 'X_tsne'


sc.pp.filter_cells(adata, min_genes=500)
sc.pp.filter_genes(adata, min_cells=10)
print(adata.n_obs, adata.n_vars)

results_file = 'adata.h5ad'
adata.write(results_file)

FULL CODE
updated version: h5ad filter, added layers to make the filtered file usable for seurat to read and make a correct seurat object
#https://github.com/satijalab/seurat/issues/2485

import scanpy as sc
import numpy as np
import pandas as pd
import time
from ipywidgets import interact, interactive, widgets
from IPython.display import display, HTML

def filter_and_save_h5ad(h5ad_file, min_genes, min_cells):
    # h5ad file read
    adata = sc.read_h5ad(h5ad_file)

    # cells and genes filter parameter from user
    sc.pp.filter_cells(adata, min_genes=min_genes)
    sc.pp.filter_genes(adata, min_cells=min_cells)

    # Add 'counts' layer
    adata.layers['counts'] = adata.X.copy()

    # add 'winsorization' layer
    adata.layers['winsorized'] = adata.X.copy()


    # filtered dataset save
    filtered_file = h5ad_file.replace('.h5ad', '_filtered.h5ad')
    adata.write(filtered_file)

    print(f"Filtered dataset saved to {filtered_file}")

# input widgets
h5ad_file_widget = widgets.Text(description='File_add:')
min_genes_widget = widgets.IntText(description='Min_Genes:')
min_cells_widget = widgets.IntText(description='Min_Cells:')
button = widgets.Button(description='Filter_Save')

# timer widget
time_widget = widgets.HTML(value="Time passed away: 0 seconds")

def on_button_clicked(click_me):
    h5ad_file = h5ad_file_widget.value.strip()
    min_genes = min_genes_widget.value
    min_cells = min_cells_widget.value
    
    # timer
    start_time = time.time()
    
    filter_and_save_h5ad(h5ad_file, min_genes, min_cells)
    
    # Calculate and update the elapsed time
    elapsed_time = time.time() - start_time
    time_widget.value = f"Time passed away: {elapsed_time} seconds"

button.on_click(on_button_clicked)

# Display the widgets and button
display(h5ad_file_widget, min_genes_widget, min_cells_widget, button, time_widget)

**Tags:** scrnaseq, single-cell

---

## 2 answers

### Answer by username_1 on 2023-05-19. Score: 1 | Accepted answer

You didn't show us how the original .h5ad object was generated, how you read it in, how you wrote it, and if you did anything else in-between besides filtering. [`AnnData.write()`](https://anndata.readthedocs.io/en/latest/generated/anndata.AnnData.write.html) takes a `compression` arg that might explain the difference. As also mentioned in that link, ensuring you are writing a sparse matrix can make a substantial difference to file size (the matrix in `adata.X` is generally by far the largest part of the object)

> **Comments:**
> **username_0**: I was taking this dataset https://data.humancellatlas.org/explore/projects/85a9263b-0887-48ed-ab1a-ddfa773727b6/get-curl-command  
> **username_0**: I have updated the question  
> **username_0**: I have updated my code please have a look and for this "You didn't show us how the original .h5ad object was generated" please have a look https://data.humancellatlas.org/explore/projects/85a9263b-0887-48ed-ab1a-ddfa773727b6  

---

### Answer by username_2 on 2023-05-18. Score: 1

Scanpy's `h5ad` is in fact hdf5 format. This format is used for rapid read/write (i/o) large amounts of data and thats its purpose. Its fairly easy to use from scratch.

Its purpose is not to compress data. If you are concerned with the contents of `h5ad` why not just read them directly via the `hdf5` viewer. You can download it here. <https://www.hdfgroup.org/downloads/hdfview/>

The file will then open up as a "Explorer" (Windows) or "Finder" (OSX) GUI file system and you can open the various files and see whats in them.

My suspicion is that the original data is been written into the file. 'cause it can i/o rapidly and access precisely what it needs then file size could be a secondary consideration. Anyway you can peer in and find out by looking at the original file and the hdf5 save you've done.

An example from the website is here

[![enter image description here](https://i.sstatic.net/zhtSf.png)](https://i.sstatic.net/zhtSf.png)

Cool right?

Personally, I think HDF5 is overrated and `feather` outperforms it for speed (i/o) - but thats just for dataframes. Its not much faster than pickling in case you are interested and pickling in addition does compression. parquet is the format that does rapid i/o but specialises in compression, again mostly for dataframes.

HDF5 however does permit very pretty GUI interfaces.

---

Also if you want to load up and look at various files, that just uses the Python `hdf5` module.


---

