pytorch dataloader to device

Clear Copperhead answered on June 4, 2021 Popularity 10/10 Helpfulness 3/10

answer pytorch dataloader to device

related pytorch dataloader

pytorch dataloader to device

Comment

You can modify the collate_fn to handle several items at once:

from torch.utils.data.dataloader import default_collate

device = torch.device('cuda:0')  # or whatever device/cpu you like

# the new collate function is quite generic
loader = DataLoader(demo, batch_size=50, shuffle=True, 
                    collate_fn=lambda x: tuple(x_.to(device) for x_ in default_collate(x)))
Note that if you want to have multiple workers for the dataloader, you'll need to add

torch.multiprocessing.set_start_method('spawn')
after your if __name__ == '__main__' (see this issue).

Having said that, it seems like using pin_memory=True in your DataLoader would be much more efficient. Have you tried this option?
See memory pinning for more information.

Popularity 10/10 Helpfulness 3/10 Language python

Source: stackoverflow.com

Tags: device python pytorch-dataloader

Link to this answer
Share Copy Link

Contributed on Jun 24 2022

Clear Copperhead

0 Answers Avg Quality 2/10

Closely Related Answers

pytorch dataloader

Comment

DataLoader(dataset, batch_size=1, shuffle=False, sampler=None,
           batch_sampler=None, num_workers=0, collate_fn=None,
           pin_memory=False, drop_last=False, timeout=0,
           worker_init_fn=None, *, prefetch_factor=2,
           persistent_workers=False)

Popularity 10/10 Helpfulness 3/10 Language python

Source: pytorch.org

Tags: python py

Link to this answer
Share Copy Link

Contributed on Jun 04 2021

Arko

0 Answers Avg Quality 2/10

pytorch dataloader

Comment

DataLoader(dataset, batch_size=1, shuffle=False, sampler=None,
           batch_sampler=None, num_workers=0, collate_fn=None,
           pin_memory=False, drop_last=False, timeout=0,
           worker_init_fn=None, *, prefetch_factor=2,
           persistent_workers=False)

Popularity 10/10 Helpfulness 2/10 Language python

Source: pytorch.org

Tags: python py

Link to this answer
Share Copy Link

Contributed on Aug 23 2022

arshi khan

0 Answers Avg Quality 2/10

pytorch dataloader to device

Contents

More Related Answers

pytorch dataloader to device

Closely Related Answers

pytorch dataloader

pytorch dataloader

Grepper

Documentation

Social

Legal

Contact

Oops, You will need to install Grepper and log-in to perform this action.