Q: Find repeated elements in list and average corresponding elements I have a list of the following type in my code:
nodes = [[n1, [x1,y1,z1], [a1,b1,...]], [n2, [x2,y2,z2], [a2,b2,..]], ...]

The elements n identify a node with a certain coordinate [x,y,z], and these can be repeated trough the list.
I wanted to average the [an,bn,...] lists for each node value n, such that I end up with a list nodes_averaged like [[[x1,y1,z1],[a1,b1,...]], [[xn,yn,zn],[an,bn,...]]]
For example, if I have the following list:
nodes= [[10, [6.5, 55.2, -10.0], [0.0, 0.0, 0.0, 0.0, 0.0, 0.0]], 
        [10, [6.5, 55.2, -10.0], [0.0, 0.0, 0.0, 0.0, 0.0, 0.0]],
        [3, [4.3, 55.4, -15.0], [-0.0016, -0.00058, -0.0256, -7.07e-06, 0.00051, 0.0088]],
        [1, [8.7, 54.9, -15.0], [-0.0016, -0.00058, -0.0256, 1.2e-05, -0.00044, 0.0088]],
        [10, [6.5, 55.2, -10.0], [-0.0011, -0.00041, -0.027, -1.12e-05, -0.00043, 0.0038]],
        [3, [4.3, 55.4, -15.0], [-0.00113, -0.000413, -0.027, 2.84e-06, 0.00039, 0.00389]]]

I wanted to obtain (the order is not important):
nodes_avg= [[[6.5, 55.2, -10.0], [-0.00036666666666666667, -0.00013666666666666666, -0.009, -3.7333333333333333e-06, -0.00014333333333333334, 0.0012666666666666666]],
            [[4.3, 55.4, -15.0], [-0.001365, -0.0004965, -0.0263, -2.115e-06, 0.00045, 0.006345]], 
            [[8.7, 54.9, -15.0], [-0.0016, -0.00058, -0.0256, 1.2e-05, -0.00044, 0.0088]]]

So far my approach was to extract a list of the node's range:
noderng = [n1, n2,..., nn]

and using node_range=list(dict.fromkeys(node_range)) to eliminate repeated nodes, then iterating over both lists to match the elements n from the nodes list to the elements in the node_range list, like this:
noderng=[]
for l in nodes:
    noderng.append(l[0])
noderng=list(dict.fromkeys(noderng))

nodeavg={}
elposition={}

for n in noderng:
    nodeavg[n]=[[],[],[],[],[],[]]
    for l in nodes:
        if int(l[0])==int(n):
            for i in range(6):
                nodeavg[n][i].append(l[2][i])
            elposition[n]=l[1]

avg={}    
for key, strains in nodeavg.items():
    avg[key]=[]
    for ess in strains:
        avg[key].append(sum(ess)/len(ess))

nodes_avg=[]
for key ,value in avg.items():
    nodes_avg.append([elposition[key],value])

I get the desired result, the problem is that the nodes list can have hundreds of thousands of elements, and it'll take hours to do this operation. I have switched to using numpy arrays, and it does help slightly, but it only shaves off a couple of minutes, which is not a great help.
Is there a more efficient way to do this operation?

A: This is not an answer just a few suggestions:

*

*You can use timer and cProfile to understand which part of your code takes more time and focus on that part.


*another suggestion that has worked for me in the past is to create a dictionary from a set that includes your keys. In this way, you can reduce the hash time. This works if you know all the n values beforehand.


*Check this to have a better understanding of the time complexity of different structs in Python.
