Fractal Networks for AI

Fractal Networks and Deep Learning

Fractal networks are increasingly the subject of cutting-edge research in many AI domains. With their complex repeating architectures at multiple scales, such networks lend themselves naturally to neural networks and bring with them significant computational advantages.

This article looks at fractal networks from the perspective of deep learning, examining their architecture, use cases, and distinctive advantages over conventional network architecture.

What Are Fractal Networks?

A fractal is an element characterized by a pattern that repeats itself across scales. Upon zooming into a fractal, it is made up of the same elements as itself, which are themselves, in turn, made of the same fractal core architecture. 

A fractal network is a network architecture that is constructed from fractal expansion principles. A functional block in a neural network might exhibit fractal architecture, consisting of fractal repetition itself at the micro and macro scales. Thus, there is an underlying symmetry in the design. 

Fractals are common motifs throughout nature: from the veins of leaves to the branches on trees, they are everywhere. Even electrical discharges have been known to exhibit fractal properties. 

Fractal Networks in The Deep Learning Ecosystem

Deep neural architectures mimic complex neuron networks in the brain. The network typically comprises several interconnected units that exhibit the behavior of biological neurons. The connections between the various units are not static and will keep changing as the network trains itself, learning throughout its operation. 

A neural network with fractal architecture will have many sub-levels with varying degrees of interconnections between the nodes. Constant communication between the different levels of the networks enables the handling of complex tasks. 

In fact, some experts have even linked this fractal model to the consciousness mechanism in the human brain. 

Why Use Fractal Networks?

The key to the power of fractal networks is that there is a single expansion rule that leads to an ultra-deep network. The network will have sub-paths of varying lengths, all interacting with each other. With the addition of appropriate nonlinearities and filters, signals undergo continuous transformation between layers. 

In contrast to usual deep networks, that might not be necessarily fractal-based, fractal networks can achieve higher complexity and exceed performance levels with low error rates. Deeply convolutional networks can be created in this manner.

Fractal networks also have certain intrinsic design advantages. An efficiency-optimized network will see efficiency maintained throughout the sub-levels. 

Typical Industry Use Cases of Fractal Neural Networks

• Computationally-Intensive Problems: It is widely believed that the greater the degree of complexity of a neural network, the more complex problems it can handle. As the depth of a self-similar network increases, its computational power increases manifold. Thus, ultra-deep fractal networks have obvious advantages in performing complex tasks as compared to conventional network architectures. 
• General Learning for Specific Tasks: The common elements and key learnings from a variety of AI sub-domains, such as speech recognition, game modeling, and computer vision can be utilized for a wide array of tasks. Fractal-based approaches facilitate this methodology.
• Fractal Machine Learning Models for Fractal Data: While conventional machine learning algorithms can be used for learning fractal data, the use of fractal algorithms can lead to powerful coupling between the data and the algorithm with these being matched to each other. Such latter algorithms will have obvious advantages over conventional ones. Thus, fractal models can be particularly computationally effective when the data at hand is fractal in character. 
• Natural Language Processing: Fractal neural networks have also been shown to be particularly effective for natural languages. Such data often has temporal dependencies in the sequences of symbols which can be too complex for conventional networks. Fractal networks can serve this purpose easily, though. 

Conclusion

Fractal networks embody the characteristics of self-similarity at every scale. Such a recursive architecture has obvious computational advantages, particularly when it comes to handling complex temporally sensitive NLP data. 

Such networks can be extremely deep in design and with every new layer comes increased computational power that is hard to obtain with an increase in the number of nodes alone. In more ways than one, fractal networks mimic natural systems and take artificial neural networks one step closer to natural biological systems.

You can reach out to us on infoindia@xebia.com for any help in this area.

------------------------------------------------------------------------------------------------------------

Disclaimer: This publication contains general information and is not intended to be comprehensive nor to provide professional advice or services. This publication is not a substitute for such professional advice or services, and it should not be acted on or relied upon or used as a basis for any investment or other decision or action that may affect you or your business. Before taking any such decision you should consult a suitably qualified professional advisor. While reasonable effort has been made to ensure the accuracy of the information contained in this publication, this cannot be guaranteed, and neither associated organization nor any affiliate thereof or other related entity shall have any liability to any person or entity which relies on the information contained in this publication. Any such reliance is solely at the user’s risk. This article may contain references to other information sources.