Tumours can escape the immune system by expressing programmed death-ligand-1 (PD-L1), which allows them to bind to PD-1 on T-cells and avoid recognition by the immune system. Regulatory T-cells (Tregs), indoleamine 2,3-dioxygenase (IDO) and tryptophan 2,3-dioxygenase (TDO) also play a role in immune suppression. Knowledge about the interaction of neuroendocrine tumours (NETs) with their immune microenvironment and the role of immunotherapy in patients with NET is scarce. Here, we investigated the immune microenvironment of serotonin-producing (SP) and non-serotonin-producing NETs (NSP-NETs). Tumours of 33 patients with SP-NET and 18 patients with NSP-NET were studied. Immunohistochemical analyses were performed for PD-L1, T-cells, IDO, TDO, mismatch repair proteins (MMRp) and activated fibroblasts. PD-L1 expression was seen in < 1% of tumour and T-cells. T-cells were present in 33% of NETs, varying between 1 and 10% T-cells per high power field. IDO was expressed in tumour cells in 55% of SP-NETs and 22% of NSP-NETs (p = 0.039). TDO was expressed in stromal cells in 64% of SP-NETs and 13% of NSP-NETs (p = 0.001). No tumours had loss of MMRp. TDO-expressing stromal cells also strongly expressed α-SMA and were identified as cancer-associated fibroblasts (CAFs). Factors that are associated with a response to checkpoint inhibitor treatment were absent or only present to a limited extent in the tumour microenvironment of NETs. The expression of IDO and TDO in a substantial part of NETs and the presence of CAFs suggest two mechanisms that could be responsible for the cold immune microenvironment, which should be explored to enhance anti-tumour immunity and clinical responses.