Figure convolution network definition and simple example

In today’s world, many critical datasets come in the form of diagrams or networks—social networks, knowledge graphs, protein interaction networks, and even the World Wide Web. While these structures are common, it wasn’t until recently that researchers began exploring how to extend traditional neural network models to effectively process such structured data. Currently, several methods have shown promising results in specialized domains. Before this, the best outcomes were achieved using kernel-based approaches, graph theory-based regularization techniques, or other similar strategies. These early methods laid the groundwork for more advanced models, especially those based on graph convolutions. Outline: - A brief introduction to neural network diagram models - Spectral Convolution and Graph Convolution Networks (GCNs) - Demo: Graph embedding with a simple first-order GCN model - Thinking of GCNs as a micro-generalization of the Weisfeiler-Lehman algorithm - If you’re already familiar with GCNs, skip to “GCNs Part III: Embedding Karate Club Network” - How powerful is the graph convolution network? Recent literature has explored ways to adapt well-established neural models like RNNs or CNNs to handle arbitrary graph structures. Some studies introduced custom architectures for specific tasks, while others built convolutional maps based on spectral theory. To define filters in multi-layer neural networks, researchers have drawn inspiration from classic CNNs. More recent work has focused on bridging the gap between fast heuristics and more theoretically grounded approaches, particularly in the spectral domain. Defferrard et al. (NIPS 2016) used Chebyshev polynomials with learnable parameters in a neural network framework, achieving impressive results on standard datasets like MNIST, comparable to simple 2D CNNs. Kipf and Welling took a similar approach but simplified the spectral convolution framework, resulting in faster training times and competitive accuracy across multiple benchmark datasets. GCNs Part I: Definitions Most graph neural network models follow a shared architecture, collectively known as Graph Convolutional Networks (GCNs). The term "convolution" refers to the fact that filter parameters are typically shared across all nodes in the graph. The goal of these models is to learn a function that maps node features into meaningful representations. The input includes a feature matrix X (N × D), where N is the number of nodes and D is the number of input features, and an adjacency matrix A representing the graph structure. The output is a node-level representation Z (N × F), where F is the number of output features per node. Each layer can be written as a non-linear transformation: H^(l+1) = σ(D^(-1)A H^(l) W^(l)) Where H^(0) = X, H^(L) = Z, and W^(l) are the weights at each layer. GCNs Part II: A Simple Example Consider a basic propagation rule: H^(l+1) = σ(D^(-1)A H^(l) W^(l)) This model, though simple, is surprisingly effective. However, two key limitations exist: 1. Multiplying by A sums features from neighbors without including the node itself unless there's a self-loop. This can be addressed by adding an identity matrix to A. 2. A is often unnormalized, which can distort feature distributions. Normalizing A ensures we're averaging neighbor features, leading to more stable behavior. Combining these adjustments gives us the propagation rule used in Kipf & Welling's work. GCNs Part III: Embedding the Karate Club Network Let’s look at Zachary’s Karate Club Network, a well-known dataset. Using a 3-layer GCN with random weights, we observe that even before training, the model generates embeddings that align with the community structure of the graph. This suggests that the model inherently captures structural information. This result is intriguing, as similar embeddings can be achieved through unsupervised methods like DeepWalk. But here, we achieve it with a simple, untrained GCN. Thinking of GCNs as a differentiable version of the Weisfeiler-Lehman algorithm helps explain this behavior. The algorithm iteratively updates node features based on their neighbors’ features, similar to what GCNs do, but in a differentiable and parameterized way. GCNs Part IV: Semi-Supervised Learning Since our model is fully differentiable, we can use semi-supervised learning. By labeling just one node per class, the model learns to propagate labels through the graph. After training, the hidden space becomes linearly separable, showing the model’s ability to capture meaningful structure without explicit feature engineering. Conclusion Research in this area is still in its early stages, but recent progress has been encouraging. While we’ve made strides, many questions remain—how to apply these models to directed graphs, or how to leverage embeddings for downstream tasks. This article only scratches the surface, and I’m excited to see what future developments will bring.

AC And DC Power Systems

AC and DC power systemsï¼›YZPST-STMS-110V-100AH(3MXG)

YANGZHOU POSITIONING TECH CO., LTD. , https://www.yzpst.com