Follow

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use
Contact

i faced this problem while i was strat sampling the dataset

In [16] : Strat_d3=d3.groupby('Label', group_keys=False).apply(lambda x: x.sample(1000))
Traceback (most recent call last):

  File "<ipython-input-16-f54910ba8f95>", line 1, in <module>
    Strat_d3=d3.groupby('Label', group_keys=False).apply(lambda x: x.sample(1000))

  File "C:\Users\Msi\anaconda3\lib\site-packages\pandas\core\groupby\groupby.py", line 894, in apply
    result = self._python_apply_general(f, self._selected_obj)

  File "C:\Users\Msi\anaconda3\lib\site-packages\pandas\core\groupby\groupby.py", line 928, in _python_apply_general
    keys, values, mutated = self.grouper.apply(f, data, self.axis)

  File "C:\Users\Msi\anaconda3\lib\site-packages\pandas\core\groupby\ops.py", line 238, in apply
    res = f(group)

  File "<ipython-input-16-f54910ba8f95>", line 1, in <lambda>
    Strat_d3=d3.groupby('Label', group_keys=False).apply(lambda x: x.sample(1000))

  File "C:\Users\Msi\anaconda3\lib\site-packages\pandas\core\generic.py", line 5350, in sample
    locs = rs.choice(axis_length, size=n, replace=replace, p=weights)

  File "mtrand.pyx", line 959, in numpy.random.mtrand.RandomState.choice

ValueError: Cannot take a larger sample than population when 'replace=False'

>Solution :

The messages means in, at least, one group you have not enough sample (< 1000).
2 solutions:

MEDevel.com: Open-source for Healthcare and Education

Collecting and validating open-source software for healthcare, education, enterprise, development, medical imaging, medical records, and digital pathology.

Visit Medevel

  1. Use replace=True to get 1000 samples but some duplicates:
# You don't need apply here
Strat_d3 = d3.groupby('Label', group_keys=False).sample(1000, replace=True)
  1. Use this trick if you accept some groups have less than 1000 samples:
Strat_d3 = d3.groupby('Label', group_keys=False).apply(lambda x: x.sample(min(len(x), 1000)))

To debug your groups, use the following code to check labels where number of samples are below 1000:

d3.value_counts('Label').loc[lambda x: x < 1000]
Add a comment

Leave a Reply

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use

Discover more from Dev solutions

Subscribe now to keep reading and get access to the full archive.

Continue reading