haystack

1
---
2
upgrade:
3
  - |
4
    Implement framework-agnostic device representations. The main impetus behind this change is to move away from stringified representations
5
    of devices that are not portable between different frameworks. It also enables support for multi-device inference in a generic manner.
6

7
    Going forward, components can expose a single, optional device parameter in their constructor (`Optional[ComponentDevice]`):
8
    ```python
9
    import haystack.utils import ComponentDevice, Device, DeviceMap
10
    class MyComponent(Component):
11
        def __init__(self, device: Optional[ComponentDevice] = None):
12
            # If device is None, automatically select a device.
13
            self.device = ComponentDevice.resolve_device(device)
14

15
        def warm_up(self):
16
            # Call the framework-specific conversion method.
17
            self.model = AutoModel.from_pretrained("deepset/bert-base-cased-squad2", device=self.device.to_hf())
18

19
    # Automatically selects a device.
20
    c = MyComponent(device=None)
21
    # Uses the first GPU available.
22
    c = MyComponent(device=ComponentDevice.from_str("cuda:0"))
23
    # Uses the CPU.
24
    c = MyComponent(device=ComponentDevice.from_single(Device.cpu()))
25
    # Allow the component to use multiple devices using a device map.
26
    c = MyComponent(device=ComponentDevice.from_multiple(DeviceMap({
27
          "layer1": Device.cpu(),
28
          "layer2": Device.gpu(1),
29
          "layer3": Device.disk()
30
    })))
31
    ```
32