Thiago Crepaldi

This is the first in a series of posts that illustrate the DuCx programming language. DuCx is a general language. It is a simplification, and a modest extension of the C language. 1/ A little more on Gedae with DuCx. This focuses on the compiler knowing about the memory required for data storage, particularly, when the memory is needed during the execution of the code, and how much is needed. It introduces range variables that specify the possible sizes of an array index. Note, arrays can be used for databases, trees, graphs and other data structures. The following DuCx code is a simple example of a matrix multiply:   range r = 10;   range i = 2;   range c = 5;   float x[r][i] = <read x>;   float y[i][c] = <read y>;   z[r][c] += x[r][i] * y[i][c]; // sum over index i The DuCx language implies the C for loops required to implement the matrix multiply. While the range sizes shown here are constant, they can also be specified as the possible set of sizes. For example, range n = {8,4,N}; meaning the value of N can range from 4 to 8-1 and the current value is N. The range sizes combined with the schedule of execution of functions and lines of DuCx code enable the compiler. It can compute the memory requirements at any point during the software execution. Another post will discuss persistent data tokens, that is, when memory is needed for repeated executions of code. The use of tokens eliminates the need for C pointers and reduces the need for "for loops". 

2

Really and ... seriously respectfully? Literally people are investing/building 10 Trillion dollars (actual money) economy in the next five years in this area that includes all AI companies, and all for just hypes... same as saying 95% fools and 5% real ! #ai #economy #hype #future

2

1 Comment

This has probably been one of the most helpful explanations for Arc and Mutex usage in Rust I've come across. Really clear examples and edge cases for a toolset that is pretty widely used in the language for concurrency. https://lnkd.in/gah7q2gw

4

<rant subject = #streaming and #queues> Streams have stronger ordering guarantees than message queues. Generally this means that all messages within a partition are read in the same order. This is highly useful when you need an ordered record of events that happened. As a consequence, streaming systems don't have individual message acknowledgement as it would violate ordering guarantees. Message queues have per-message acknowledgement, which means they have less strict ordering guarantees. One hybrid approach which is quite cool is "key-based subscriptions" in Apache Pulsar and also Confluent's Parallel Consumer for #apachekafka which guarantee order for all messages with the same key while allowing you to individually acknowledge messages. The tradeoff with individual message ack's is persistence (disk or memory)—you need to keep track of which messages have been acknowledged and which ones are still inflight. Pulsar uses BookKeeper to persist "subscriptions." Parallel Consumer stores them in the user-defined metadata in the consumer offset topic. There's an in-flight KIP for "Queues on Kafka" (KIP-932) but if I were the Grumpy Maintainer of Kafka, I would say that it's unnecessary given that it's already possible to implement queue semantics on top of Kafka in the client side (see: Parallel Consumer). Why add additional code bloat on the broker side when the features are already accessible? </rant>

4

I reopened yesterday "Understanding the Linux Kernel, 3rd Edition," (which I used to read episodically in the past) to quickly browse the VFS chapter, and to my surprise, I found it still relevant for kernels +5.x with most VFS structure names almost the same after setting up Visual Studio Code to browse specific versions of kernel source code with IntelliSense. Despite nearly 20 years passed since the publication of this book, I think it is still very good for a quick conceptual understanding of Linux internals in terms of structure relationships, even if some structures differ wildly due to restructuring (no pun intended). It can also be read very quickly because various excruciating details can now be safely skipped :-)

553

17 Comments

With work on real-time preemption largely complete for the linux kernel, new work on overhauling non-realtime preemption is beginning. The end goal is to have a scheduler with only two non-realtime modes: PREEMPT_LAZY and PREEMPT_FULL. The [new ] lazy mode will occupy a place between PREEMPT_NONE and PREEMPT_VOLUNTARY, replacing both of them. https://lnkd.in/gBrp7BTN #linux

1

Another issue of Humans of the Data Sphere is out! In this issue we also explore compute-compute separation in modern distributed data systems. https://lnkd.in/drMtXMiR

6

With conditional writes, S3 takes one more step towards the universal storage layer. But can it go the next level? Is it possible to make a (profitable) storage service that hits all three needs of latency, cost, and durability?

13

The thing is, systems have traditionally been designed around paying for capacity of network and disks, not the number of IO requests. That fundamentally changes with S3, and right now, I don't think that cost model is compatible with S3 being a universal storage layer. What it does is make system designers twist themselves up in knots to make a system with good economics. Cross-AZ charges, charging per 1000 IO requests, it fundamentally changes how you build software in the cloud. It also introduces a certain amount of design volatility as cloud pricing changes. Everyone is playing cloud arbitrage to build profitable data infra companies on the hyperscalers. One thing I've learned is that if you want to be successful over the long term, you need to build your systems to be flexible to the changing pricing models of the cloud providers. If you don't, a plucky little start-up will.

12

2 Comments

If we give Robots "Working Memory" systems, how large should their WM buffers be, and how long should entities remain in WM before decaying away? It depends on context. Rafael Sousa Silva explores in his Cogsci 2024 paper: https://lnkd.in/gx4fquHJ In this work Rafael examines robot WM systems that are not bound to human-like restrictions, where every task-relevant entity can have its own cache of relevant features. But just because robots have vast memories doesn't mean these systems don't need limits! In this work, Rafael carefully examines how the upper and lower bounds on both decay and interference parameters are sensitive to (1) contextual factors, and (2) tolerance for risk. Specifically, Rafael analyzes the error modes of WM systems -- what can go wrong if decay and interference parameters are too low or too high? For example, if decay is too large, it becomes increasingly probably that WM will contain "stale" properties that are no longer accurate. If we assume properties go stale following an exponential distribution, we can precisely analyze the relationship between decay rates and our risk tolerance for property staleness! Rafael performs this type of analysis for the upper and lower bounds of interference and decay parameters, ultimately allowing him to formulate "Metarules" for simultaneously reasoning about these upper and lower bounds. This was a surprisingly fun paper -- and one that REALLY benefitted from this year's CogSci reviews, which encouraged Rafa to really flesh out some of this math for the paper's final version. To learn more, read the paper! https://lnkd.in/gx4fquHJ And big thanks to National Science Foundation (NSF) for funding it!

29

Real-time Linux is finally here, but long before it got here the project had made fundamental improvements to mainstream Linux.

26

Interesting to see the use cases this opens up especially in automotive and robotics. Previously where real time performance is a concern, Linux-based controllers are almost always out of the table. I also wonder if this will completely displace embedded non-Linux and alternatives like RTOS-driven microcontrollers for non low power appplications.

1

Linux Kernel 6.12 Confirmed As LTS, Will Be Supported For 'Multiple Years': Slashdot reader prisoninmate shared this report from the blog 9to5Linux Renowned Linux kernel developer Greg Kroah-Hartman announced Thursday that the Linux 6.12 kernel series has been officially marked as LTS (Long Term Support) on the kernel.org website with a predicted life expectancy of at least two years. Linux kernel 6.12 was released on November 17th, 2024, and introduces new features like real-time "PREEMPT_RT" support, a new scheduler called sched_ext, and DRM panic messages as QR codes, as well as numerous new and updated drivers for better hardware support... Linux kernel 6.12 joins the many other long-term support kernel branches, namely Linux 6.6 LTS, Linux 6.1 LTS, Linux 5.15 LTS, Linux 5.10 LTS, and Linux 5.4 LTS. Apart from the latter, the rest of them, including Linux kernel 6.12, will be officially supported until the end of December 2026. Hopefully, Linux kernel 6.12 will be supported for more than two years as the kernel maintainers usually aim for four years of support for a new LTS kernel, especially if there's demand from hardware manufacturers and other companies that aim to use a long-term supported kernel in their devices. Read more of this story at Slashdot.

1

Visual explanation of transformers...

3

There's a large part of the EDA community still behaves like it's pre 2000, and insists on doing batch processing, assisted by folks that push methodologies like UVM/CR which eat up resources doing things better handled by formal verification. There are better ways to do EDA, but you won't get them from folks who get paid for license hours with point-tools, or Cloud computing resources. https://lnkd.in/g9akrCtU

1

What is NVDA NIM? I'm deeply researching Nvidia products this week; it's amazing how many complex techs they have under their belt.  1. NIM (NVIDIA Inference Microservices) allows AI models to be run on various NVIDIA GPUs (cloud, data center, workstations, PCs). 2. Built for cloud-native environments, NIM offers easy deployment (single command) and scales with Kubernetes. 3. It integrates with enterprise management tools through identity, metrics, and health checks. 4. Powered by inference engines (Triton Inference Server, TensorRT, TensorRT-LLM), NIM simplifies model deployment for large-scale AI inference. 5. Standard APIs enable developers to build enterprise-grade AI applications. #AI, #Founder, #Startups, #SoftwareEngeniering For more insights about AI Engineering and Startups, check out my newsletter: 🚀founderio.substack.com🚀

2

1 Comment

About 6 years ago I made a presentation called "messaging, the second hardest thing to trace" because I felt at the time that engineers cherry-pick the easy parts. Those doing the hard parts rarely had good help from #opensource projects at the time. This made me concerned and while the scenario changes the sentiment remains the same. We desire simplicity and usually that means cutting out the hard parts. Let's remember folks 50+ years ago put people on the moon... we can do the hard parts. It is exciting to do the easy parts in a new tech area, but someone needs to do the hard parts. Is it you? If so, please talk about it publicly. We need to romanticize the hard parts as that's what keeps tech working.

9

1 Comment

Can Differential Transformer solve the attention noise issue in long-context LLMs? Just several weeks after NVIDIA published a paper on how LLMs struggle at the limits of their context windows (which I discussed in my last post: https://lnkd.in/ghW3ZdK4), researchers from Microsoft Research and Tsinghua University introduced a potential solution: the 𝗗𝗶𝗳𝗳𝗲𝗿𝗲𝗻𝘁𝗶𝗮𝗹 𝗧𝗿𝗮𝗻𝘀𝗳𝗼𝗿𝗺𝗲𝗿. Traditional Transformers can handle large sequences, but they often suffer from 𝗮𝘁𝘁𝗲𝗻𝘁𝗶𝗼𝗻 𝗱𝗶𝘀𝗽𝗲𝗿𝘀𝗶𝗼𝗻 as the context size increases. The standard attention mechanism tends to spread focus too thin, over-allocating attention to irrelevant tokens, which makes it difficult for the model to pinpoint and prioritize the most relevant information. This results in a drop in accuracy, especially when key details are buried within massive contexts. The Differential Transformer aims to fix this by working like an 𝗮𝘁𝘁𝗲𝗻𝘁𝗶𝗼𝗻 𝗮𝗺𝗽𝗹𝗶𝗳𝗶𝗲𝗿. It calculates attention as the difference between two separate softmax maps, producing a sparser and more precise attention pattern. This is similar to how differential amplifiers in electrical engineering cancel out common-mode noise by using the difference between two signals. This approach can 𝗯𝗼𝗼𝘀𝘁 𝗮𝗰𝗰𝘂𝗿𝗮𝗰𝘆 𝘂𝗽 𝘁𝗼 𝟳𝟲% when working with contexts up to 64K tokens. What’s even more remarkable is that it matches FLOPs and parameter counts with the original Transformer, achieving these improvements 𝘄𝗶𝘁𝗵𝗼𝘂𝘁 𝗲𝘅𝘁𝗿𝗮 𝗰𝗼𝗺𝗽𝘂𝘁𝗮𝘁𝗶𝗼𝗻𝗮𝗹 𝗰𝗼𝘀𝘁. Will this technique bring language models to the next level of scalability, like how 𝗿𝗲𝘀𝗶𝗱𝘂𝗮𝗹 𝗰𝗼𝗻𝗻𝗲𝗰𝘁𝗶𝗼𝗻𝘀 enabled deeper networks or how 𝗱𝗿𝗼𝗽𝗼𝘂𝘁 helped reduce overfitting? Only time will tell, but it’s certainly an exciting development to watch. Full details in the paper linked in the comments. #MachineLearning #ArtificialIntelligence #AI #LLM #GenAI #NeuralNetworks

84

7 Comments