<p>The following would be useful,</p>
<pre><code>print (heterozygosity_df.columns)
</code></pre>
<p>This looks like static typing issue within pandas. What I suspect is <code>heterozygosity_df.['chrI']</code> is a column in the dataframe. What I think has happened is there's a mix of strings and floats within this column. pandas has set this as a &quot;string&quot; but you are wanting to perform numerical operations. Thus the solution is simply</p>
<pre><code>print(heterozygosity_df.dtypes) # this should state 'chrI' is a &quot;category&quot; or &quot;string&quot;
heterozygosity_df['chrI'] = heterozygosity_df['chrI'].astype(float)
print(heterozygosity_df.dtypes)
</code></pre>
<p>If you have multiple changes the syntax is</p>
<pre><code>heterozygosity_df = heterozygosity_df.astype({'chrI':'float', 'egColumn':'category'})
</code></pre>
<p>I suspect there will be other errors, e.g. the header is the first row of the data column. This is because again for pandas to automatically assign a column to a &quot;string&quot; means there must be a string value within the column.</p>
<hr />
<p>From the comments. I see whats happening. The easy solution is ...</p>
<pre><code>replacement = {
    &quot;chrI&quot;: 1,
    &quot;chrII&quot;: 2,
    &quot;chrIII&quot;: 3,
    &quot;chrIV&quot;: 4,
    ....
}

heterozygosity_df['chr'] = heterozygosity_df['chr'].str.replace(replacement, regex=True)
</code></pre>
<hr />
<p>From the comments ... good it works ... this is what I would have personally done ..</p>
<pre><code>replacement = {
    &quot;chrI&quot;: 1,
    &quot;chrII&quot;: 2,
    &quot;chrIII&quot;: 3,
    &quot;chrIV&quot;: 4, # continue for all chromosomes
}

heterozygosity_df = pd.read_csv(&quot;file.tsv&quot;, sep=&quot;\t&quot;, header=None).set_axis(['chr', 'pos', 'het'], axis=1, copy=False)
heterozygosity_df['chr'] = heterozygosity_df['chr'].str.replace(replacement, regex=True).astype('int')
</code></pre>
<p>Normally <code>str</code> should be in place because when <code>'chr'</code> is imported its an object. However, if it works thats the only thing that counts. </p>

CC BY-SA 4.0
2023-10-03T19:16:55.757Z
when I do `print(heterozygosity_df.columns)` it outputs: `Index(['chr', 'pos', 'het'], dtype='object')` and when I do `print(heterozygosity_df.dtypes)` it outputs: `chr object    pos  int64    het  int64    dtype: object`
rimo
CC BY-SA 4.0
2023-10-03T19:19:37.247Z
Since I have chromosomes going from chrI to chrXVI I grouped the chromosomes by "chr" instead of individual "chrI" so when I did `heterozygosity_df['chr'] = heterozygosity_df['chr'].astype(float)` it just gave me the same `ValueError: could not convert string to float: 'chrI'`
rimo
CC BY-SA 4.0
2023-10-03T19:29:31.613Z
the annoying part is that I'm not using the 'chr' column to do the actual numerical operations - I just use it to group the data in the beginning (see: `for chr_heterozygosity_df in heterozygosity_df.groupby('chr'):`) the actual math goes into the position and heterozygosity columns
rimo
CC BY-SA 4.0
2023-10-03T19:38:59.920Z
@rimo Addressed below the "-----"
M__
CC BY-SA 4.0
2023-10-03T19:54:34.837Z
Traceback (most recent call last):
  File "fixing_error.py", line 90, in <module>
    heterozygosity_df['chr'] = heterozygosity_df['chr'].str.replace(replacement, regex=True)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/pandas/core/strings/accessor.py", line 129, in wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: StringMethods.replace() missing 1 required positional argument: 'repl'
rimo
CC BY-SA 4.0
2023-10-03T19:54:55.523Z
I get this error message when I implement the code you wrote above @M__
rimo
CC BY-SA 4.0
2023-10-03T20:06:26.353Z
wait I got it to kind of work by taking out the '.str' and leaving the command like this: `heterozygosity_df['chr'] = heterozygosity_df['chr'].replace(replacement, regex=True)`
rimo
CC BY-SA 4.0
2023-10-03T20:06:50.037Z
however now it's only printing chromosomes 1, 5, 10, and 17 (chrmt) for whatever reason and not the rest of them???
rimo
CC BY-SA 4.0
2023-10-03T18:23:21.107Z
2023-10-03T20:25:34.827Z
2023-10-03T20:25:34.827Z
M__
<p>Ah! I finally got it to work! I used your command but I had to change it up a little bit to work with my data. The final command I used is this:</p>
<pre><code>replacement = {
    &quot;chrI&quot;: 1,
    &quot;chrII&quot;: 2,
    &quot;chrIII&quot;: 3,
    &quot;chrIV&quot;: 4,
    &quot;chrV&quot;: 5,
    &quot;chrVI&quot;: 6,
    &quot;chrVII&quot;: 7,
    &quot;chrVIII&quot;: 8,
    &quot;chrIX&quot;: 9,
    &quot;chrX&quot;: 10,
    &quot;chrXI&quot;: 11,
    &quot;chrXII&quot;: 12,
    &quot;chrXIII&quot;: 13,
    &quot;chrXIV&quot;: 14,
    &quot;chrXV&quot;: 15,
    &quot;chrXVI&quot;: 16,
    &quot;chrmt&quot;: 17
}

heterozygosity_df['chr'] = heterozygosity_df['chr'].replace(replacement, regex=False)
</code></pre>
<p>And with that I was able to generate plots showing all of the chromosomes! (I have to figure out how to change the axis labels from 1,2,3,4...10 back to the chromosome but that's a future problem). Thank you so much for all your help!!</p>

CC BY-SA 4.0
2023-10-03T20:26:48.403Z
`regex=False` you might be right.
M__
CC BY-SA 4.0
2023-10-03T20:22:52.803Z
2023-10-03T20:22:52.803Z
rimo
<p>I'm trying to use a Python script to plot with a rolling window using pandas and seaborn. This code worked for the longest time but now it's giving me an error that I don't know how to fix.</p>
<p>Here is the code I'm using:</p>
<pre class="lang-py prettyprint-override"><code>import pandas as pd
import matplotlib.pyplot as plt
import numpy as np
import seaborn as sns

heterozygosity_df = pd.read_csv(&quot;file.tsv&quot;, sep=&quot;\t&quot;, header=None).set_axis(['chr', 'pos', 'het'], axis=1, copy=False)

heterozygosity_df['het'] = heterozygosity_df['het'].transform(lambda x: -1*(x-1))

windows = [25, 50, 100, 200]
steps = [5]

for window in windows:
    for step in steps:
        heterozygosity_ratio = {}
        for chr_heterozygosity_df in heterozygosity_df.groupby('chr'):
            heterozygosity_ratio[chr_heterozygosity_df[0]] = chr_heterozygosity_df[1].set_index('pos').rolling(window=window, step=step, center=True).mean()
...
</code></pre>
<p>There is more after this but that's just to configure the resulting plot with matplotlib. The error I get when I use the above code is this:</p>
<pre class="lang-py prettyprint-override"><code>Traceback (most recent call last):
  File &quot;/opt/homebrew/lib/python3.11/site-packages/pandas/core/window/rolling.py&quot;, line 368, in _prep_values
    values = ensure_float64(values)
             ^^^^^^^^^^^^^^^^^^^^^^
  File &quot;pandas/_libs/algos_common_helper.pxi&quot;, line 42, in pandas._libs.algos.ensure_float64
ValueError: could not convert string to float: 'chrI'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File &quot;/opt/homebrew/lib/python3.11/site-packages/pandas/core/window/rolling.py&quot;, line 485, in _apply_blockwise
    arr = self._prep_values(arr)
          ^^^^^^^^^^^^^^^^^^^^^^
  File &quot;/opt/homebrew/lib/python3.11/site-packages/pandas/core/window/rolling.py&quot;, line 370, in _prep_values
    raise TypeError(f&quot;cannot handle this type -&gt; {values.dtype}&quot;) from err
TypeError: cannot handle this type -&gt; object

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File &quot;plot_heterozygosity_window.py&quot;, line 29, in &lt;module&gt;
    heterozygosity_ratio[chr_heterozygosity_df[0]] = chr_heterozygosity_df[1].set_index('pos').rolling(window=window, step=step, center=True).mean()
                                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File &quot;/opt/homebrew/lib/python3.11/site-packages/pandas/core/window/rolling.py&quot;, line 2111, in mean
    return super().mean(
           ^^^^^^^^^^^^^
  File &quot;/opt/homebrew/lib/python3.11/site-packages/pandas/core/window/rolling.py&quot;, line 1507, in mean
    return self._apply(window_func, name=&quot;mean&quot;, numeric_only=numeric_only)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File &quot;/opt/homebrew/lib/python3.11/site-packages/pandas/core/window/rolling.py&quot;, line 615, in _apply
    return self._apply_blockwise(homogeneous_func, name, numeric_only)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File &quot;/opt/homebrew/lib/python3.11/site-packages/pandas/core/window/rolling.py&quot;, line 487, in _apply_blockwise
    raise DataError(
pandas.errors.DataError: Cannot aggregate non-numeric type: object
</code></pre>
<p>I have tried converting the 'chr' column directly from a string to a float:
<code>heterozygosity_df['chr'] = heterozygosity_df['chr'].astype(float)</code></p>
<p>I have tried converting the entire dataframe to a float:
<code>heterozygosity_df = heterozygosity_df.astype(float, errors = 'raise')</code></p>
<p>But everything keeps giving me the <code>ValueError: could not convert string to float: 'chrI'</code></p>
<p>Does someone know an easy way to fix this? Like I said, I have been using this code for months, and it's always worked - have they deprecated something in the pandas package to make it so you can't read in columns with strings and integers?</p>
<p><strong>EDIT</strong>:</p>
<p>For reference the dataframe is structured like this:</p>
<pre><code>chrI  123  1
chrI  234  0
chrI  237  1
chrII 456  1 
chrII 567  1 
chrII 678  1 
...
chrXVI 123 1
</code></pre>
<p>Where the first column designates the chromosome, the second column is the position, and the third column is the heterozygosity indicator.</p>

<python><errors><pandas>
ValueError: count not convert string to float
# ValueError: count not convert string to float

(Asked by: username_0 on 2023-10-03)

I'm trying to use a Python script to plot with a rolling window using pandas and seaborn. This code worked for the longest time but now it's giving me an error that I don't know how to fix.
Here is the code I'm using:
import pandas as pd
import matplotlib.pyplot as plt
import numpy as np
import seaborn as sns

heterozygosity_df = pd.read_csv("file.tsv", sep="\t", header=None).set_axis(['chr', 'pos', 'het'], axis=1, copy=False)

heterozygosity_df['het'] = heterozygosity_df['het'].transform(lambda x: -1*(x-1))

windows = [25, 50, 100, 200]
steps = [5]

for window in windows:
    for step in steps:
        heterozygosity_ratio = {}
        for chr_heterozygosity_df in heterozygosity_df.groupby('chr'):
            heterozygosity_ratio[chr_heterozygosity_df[0]] = chr_heterozygosity_df[1].set_index('pos').rolling(window=window, step=step, center=True).mean()
...

There is more after this but that's just to configure the resulting plot with matplotlib. The error I get when I use the above code is this:
Traceback (most recent call last):
  File "/opt/homebrew/lib/python3.11/site-packages/pandas/core/window/rolling.py", line 368, in _prep_values
    values = ensure_float64(values)
             ^^^^^^^^^^^^^^^^^^^^^^
  File "pandas/_libs/algos_common_helper.pxi", line 42, in pandas._libs.algos.ensure_float64
ValueError: could not convert string to float: 'chrI'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/homebrew/lib/python3.11/site-packages/pandas/core/window/rolling.py", line 485, in _apply_blockwise
    arr = self._prep_values(arr)
          ^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/pandas/core/window/rolling.py", line 370, in _prep_values
    raise TypeError(f"cannot handle this type -> {values.dtype}") from err
TypeError: cannot handle this type -> object

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "plot_heterozygosity_window.py", line 29, in 
    heterozygosity_ratio[chr_heterozygosity_df[0]] = chr_heterozygosity_df[1].set_index('pos').rolling(window=window, step=step, center=True).mean()
                                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/pandas/core/window/rolling.py", line 2111, in mean
    return super().mean(
           ^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/pandas/core/window/rolling.py", line 1507, in mean
    return self._apply(window_func, name="mean", numeric_only=numeric_only)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/pandas/core/window/rolling.py", line 615, in _apply
    return self._apply_blockwise(homogeneous_func, name, numeric_only)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/pandas/core/window/rolling.py", line 487, in _apply_blockwise
    raise DataError(
pandas.errors.DataError: Cannot aggregate non-numeric type: object

I have tried converting the 'chr' column directly from a string to a float:
heterozygosity_df['chr'] = heterozygosity_df['chr'].astype(float)
I have tried converting the entire dataframe to a float:
heterozygosity_df = heterozygosity_df.astype(float, errors = 'raise')
But everything keeps giving me the ValueError: could not convert string to float: 'chrI'
Does someone know an easy way to fix this? Like I said, I have been using this code for months, and it's always worked - have they deprecated something in the pandas package to make it so you can't read in columns with strings and integers?
EDIT:
For reference the dataframe is structured like this:
chrI  123  1
chrI  234  0
chrI  237  1
chrII 456  1 
chrII 567  1 
chrII 678  1 
...
chrXVI 123 1

Where the first column designates the chromosome, the second column is the position, and the third column is the heterozygosity indicator.

**Tags:** python, errors, pandas

---

## 2 answers

### Answer by username_1 on 2023-10-03. Score: 1 | Accepted answer

The following would be useful,
print (heterozygosity_df.columns)

This looks like static typing issue within pandas. What I suspect is heterozygosity_df.['chrI'] is a column in the dataframe. What I think has happened is there's a mix of strings and floats within this column. pandas has set this as a "string" but you are wanting to perform numerical operations. Thus the solution is simply
print(heterozygosity_df.dtypes) # this should state 'chrI' is a "category" or "string"
heterozygosity_df['chrI'] = heterozygosity_df['chrI'].astype(float)
print(heterozygosity_df.dtypes)

If you have multiple changes the syntax is
heterozygosity_df = heterozygosity_df.astype({'chrI':'float', 'egColumn':'category'})

I suspect there will be other errors, e.g. the header is the first row of the data column. This is because again for pandas to automatically assign a column to a "string" means there must be a string value within the column.

From the comments. I see whats happening. The easy solution is ...
replacement = {
    "chrI": 1,
    "chrII": 2,
    "chrIII": 3,
    "chrIV": 4,
    ....
}

heterozygosity_df['chr'] = heterozygosity_df['chr'].str.replace(replacement, regex=True)


From the comments ... good it works ... this is what I would have personally done ..
replacement = {
    "chrI": 1,
    "chrII": 2,
    "chrIII": 3,
    "chrIV": 4, # continue for all chromosomes
}

heterozygosity_df = pd.read_csv("file.tsv", sep="\t", header=None).set_axis(['chr', 'pos', 'het'], axis=1, copy=False)
heterozygosity_df['chr'] = heterozygosity_df['chr'].str.replace(replacement, regex=True).astype('int')

Normally str should be in place because when 'chr' is imported its an object. However, if it works thats the only thing that counts.

> **Comments:**
> **username_0**: when I do `print(heterozygosity_df.columns)` it outputs: `Index(['chr', 'pos', 'het'], dtype='object')` and when I do `print(heterozygosity_df.dtypes)` it outputs: `chr object    pos  int64    het  int64    dtype: object`  
> **username_0**: Since I have chromosomes going from chrI to chrXVI I grouped the chromosomes by "chr" instead of individual "chrI" so when I did `heterozygosity_df['chr'] = heterozygosity_df['chr'].astype(float)` it just gave me the same `ValueError: could not convert string to float: 'chrI'`  
> **username_0**: the annoying part is that I'm not using the 'chr' column to do the actual numerical operations - I just use it to group the data in the beginning (see: `for chr_heterozygosity_df in heterozygosity_df.groupby('chr'):`) the actual math goes into the position and heterozygosity columns  
> **username_1**: @rimo Addressed below the "-----"  
> **username_0**: Traceback (most recent call last):   File "fixing_error.py", line 90, in <module>     heterozygosity_df['chr'] = heterozygosity_df['chr'].str.replace(replacement, regex=True)                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^   File "/opt/homebrew/lib/python3.11/site-packages/pandas/core/strings/accessor.py", line 129, in wrapper     return func(self, *args, **kwargs)            ^^^^^^^^^^^^^^^^^^^^^^^^^^^ TypeError: StringMethods.replace() missing 1 required positional argument: 'repl'  
> **username_0**: I get this error message when I implement the code you wrote above @M__  
> **username_0**: wait I got it to kind of work by taking out the '.str' and leaving the command like this: `heterozygosity_df['chr'] = heterozygosity_df['chr'].replace(replacement, regex=True)`  
> **username_0**: however now it's only printing chromosomes 1, 5, 10, and 17 (chrmt) for whatever reason and not the rest of them???  

---

### Answer by username_0 on 2023-10-03. Score: 2

Ah! I finally got it to work! I used your command but I had to change it up a little bit to work with my data. The final command I used is this:
replacement = {
    "chrI": 1,
    "chrII": 2,
    "chrIII": 3,
    "chrIV": 4,
    "chrV": 5,
    "chrVI": 6,
    "chrVII": 7,
    "chrVIII": 8,
    "chrIX": 9,
    "chrX": 10,
    "chrXI": 11,
    "chrXII": 12,
    "chrXIII": 13,
    "chrXIV": 14,
    "chrXV": 15,
    "chrXVI": 16,
    "chrmt": 17
}

heterozygosity_df['chr'] = heterozygosity_df['chr'].replace(replacement, regex=False)

And with that I was able to generate plots showing all of the chromosomes! (I have to figure out how to change the axis labels from 1,2,3,4...10 back to the chromosome but that's a future problem). Thank you so much for all your help!!

> **Comments:**
> **username_1**: `regex=False` you might be right.  

---

