Bilimsel hesaplamada paralelleştirme ile hızlanamayan ünlü problemler / algoritmalar var mı?

Bilimsel hesaplamada paralelleştirme ile hızlanamayan ünlü problemler / algoritmalar var mı? CUDA ile ilgili kitap okurken çoğu şeyin olabileceği anlaşılıyor.

Temel sorun, kritik yol uzunluğu hesaplama toplam miktarına göre , T . Eğer C ile orantılıdır T iyi bir sabit hız-up, daha sonra paralellik teklifler. Eğer Cı daha asimptotik küçük olan T , sorun büyüklüğü arttıkça daha fazla paralellik için oda vardır. Ki burada algoritmalar için , T giriş büyüklüğü polinom , N , en iyi durumda Cı ~ günlük T çok az faydalı miktarlarda logaritmik süresinden daha kısa bir süre içinde hesaplanabilir, çünkü.$C$$T$$C$$T$$C$$T$$T$$N$$C \sim \log T$

Örnekler

• $C = T$Standart algoritmayı kullanarak bir tridiagonal çözme içinHer işlem önceki işlemin tamamlanmasına bağlıdır, bu yüzden paralellik için fırsat yoktur. Tridiagonal problemler, logaritmik bir sürede paralel bir bilgisayarda, doğrudan diseksiyonlu direk çözme, çok düzeyli alan ayrıştırması veya harmonik uzatma kullanılarak oluşturulan temel fonksiyonlara sahip çok çekirdekli bir bilgisayarda çözülebilir (bu üç algoritma, birden fazla boyutta farklıdır, ancak 1D'de tam olarak çakışabilir).
• Yoğun bir alt üçgen bir ile çözmek matrisi vardır , T = K = O ( m, 2 ) , ancak kritik yol sadece Cı = m = $m\times m$$T = N = \mathcal O(m^2)$ , bu yüzden bazı paralellikler yararlı olabilir.$C = m = \sqrt T$
• Multigrid ve FMM'nin her ikisi de , kritik yol uzunluğu C = log T'dir .$T = N$$C = \log T$
• Alanın düzenli bir bir süre için açık dalga yayılımı ( 0 , 1 ) d , k = τ / Δ t ∼ τ N 1 / d zaman adımlarını gerektirir (kararlılık için), bu nedenle kritik yol en az C = k . Toplam çalışma miktarı T = k N = τ N ( d + 1 ) / d'dir . Maksimum yararlı işlemci sayısı P = $\tau$$(0,1)^d$$k = \tau / \Delta t \sim \tau N^{1/d}$$C = k$$T = k N = \tau N^{(d+1)/d}$ , kalan faktör N 1 / d artan paralellik ile geri kazanılamaz.$P = T/C = N$$N^{1/d}$

Biçimsel karmaşıklık

Kuzey Carolina karmaşıklığı sınıfı (polylogarithmic zamanda, örneğin,) paralel olarak etkin bir şekilde çözülebilir bu sorunları karakterize etmektedir. olup olmadığı bilinmemektedir , ancak yanlış olduğu yaygın olarak varsayılmaktadır. Eğer gerçekten durum buysa, P-complete , "içsel olarak sıralı" olan ve paralellikten dolayı önemli ölçüde hızlanamayan sorunları karakterize eder.$NC = P$

$NC$$O(log^c n)$$O(n^k)$ parallel processors. It is still unknown whether $P=NC$ (although most people suspect it's not) where $P$ is the set of problems solvable in polynomial time. The "hardest" problems to parallelize are known as $P$-complete problems in the sense that every problem in $P$ can be reduced to a $P$-complete problem via $NC$ reductions. If you show that a single $P$-complete problem is in $NC$, you prove that $P=NC$ (although that's probably false as mentioned above).

So any problem that is $P$-complete would intuitively be hard to parallelize (although big speedups are still possible). A $P$-complete problem for which we don't have even very good constant factor speedups is Linear Programming (see this comment on OR-exchange).

Start by grocking Amdahl's Law. Basically anything with a large number of serial steps will benefit insignificantly from parallelism. A few examples include parsing, regex, and most high-ratio compression.

Aside from that, the key issue is often a bottleneck in memory bandwidth. In particular with most GPU's your theoretical flops vastly outstrip the amount of floating point numbers you can get to your ALU's, as such algorithms with low arithmetic intensity (flops / cache-miss) will spend a vast majority of time waiting on RAM.

Lastly, any time that a piece of code requires branching, it is unlikely to get good performance, as ALU's typically outnumber logic.

In conclusion, a really simple example of something that would be hard to get a speed gain from a GPU is simply counting the number of zeros in a array of integers, as you may have to branch often, at most perform 1 operation (increment by one) in the case that you find a zero, and make at least one memory fetch per operation.

An example free of the branching problem is to compute a vector which is the cumulative sum of another vector. ( [1,2,1] -> [1,3,4] )

I don't know if these count as "famous" but there is certainly a large number of problems that parallel computing will not help you with.

The (famous) fast marching method for solving the Eikonal equation cannot be sped up by parallelization. There are other methods (for example fast sweeping methods) for solving the Eikonal equation that are more amenable to parallelization, but even here the potential for (parallel) speedup is limited.

The problem with the Eikonal equation is that the flow of information depends on the solution itself. Loosely speaking, the information flows along the characteristics (i.e. light rays in optics), but the characteristics depend on the solution itself. And the flow of information for the discretized Eikonal equation is even worse, requiring additional approximations (like implicitly present in fast sweeping methods) if any parallel speedup is desired.

To see the difficulties for parallelization, imagine a nice labyrinth like in some of the examples on Sethian's webpage. The number of cells on the shortest path through the labyrinth (probably) is a lower bound for the minimal number of steps/iterations of any (parallel) algorithm solving the corresponding problem.

(I write "(probably) is", because lower bounds are notoriously difficult to prove, and often require some reasonable assumptions on the operations used by an algorithm.)

Another class of problems that are hard to parallelize in practice are problems sensitive to rounding errors, where numerical stability is achieved by serialization.

Consider for example the Gram–Schmidt process and its serial modification. The algorithm works with vectors, so you might use parallel vector operations, but that does not scale well. If the number of vectors is large and the vector size is small, using parallel classical Gram–Schmidt and reorthogonalization might be stable and faster than single modified Gram–Schmidt, although it involves doing several times more work.