The Next Evolution For Robotic Technology

Thoughts And Observations about Robots and Automation:

“…(W)hat’s changed more recently is the notion that robots can be designed and can adapt to more than just a few tasks. The training of robots using modeling techniques that are either borrowed from or very similar to those for large language models and even the use of large language models themselves as components of robotic systems have opened up new vistas for robots to take a particular task, learn it, improve it, and then move on to additional tasks. And the notion that robots have become more and more general purpose has really drawn a lot more interest, innovation, and investment into the robotics sector, and we’re going to see an ongoing blooming of innovation in this domain.” Roger Roberts – U.S. business consultant, partner with McKinsey & Company posted on the McKinsey Technology Trends Outlook, released September 5, 2024.

“The May (2024) program for the International Conference on Robotics and Automation, a popular event for robotics researchers, lists almost two dozen papers that involve use of vision language models. Investors are pouring money into startups aiming to apply advances in AI to robotics. … (The company) Physical Intelligence, which received an initial $70 million in funding … is working to combine large language models with real-world training to give robots general problem-solving abilities. Skild AI, founded by roboticists at Carnegie Mellon University, has a similar goal. This month it announced $300 million in funding.  Just a few years ago, a robot would need a map of its environment and carefully chosen commands to navigate successfully. Large language models contain useful information about the physical world, and newer versions that are trained on images and video as well as text, known as vision language models, can answer questions that require perception.” Posted to WIRED on July 11, 2024 by Will Knight – U.S. journalist.

“(A) robot is a very complex system and only as good as its weakest link. If the vision subsystem has a hard time perceiving what’s in front of it in direct sunlight, then the robots may suddenly go blind and stop working if a ray of sun comes through a window. If the navigation subsystem doesn’t understand stairs, then the robot may tumble down them and hurt itself (and possibly innocent bystanders). And so on. Building a robot that can live and work alongside us is hard. Like, really hard. For decades people have been trying to program various forms of robots to perform even simple tasks, like grasping a cup on a table or opening a door, and these programs have always ended up becoming extremely brittle, failing at the slightest change in conditions or variations in the environment. Why? Because of the lack of predictability in the real world (like that ray of sunlight). And we haven’t even gotten to the hard stuff yet, like moving through the messy and cluttered spaces where we live and work. Once you start thinking carefully about all this, you realize that unless you lock everything down, really tight, with all objects being in fixed, predefined locations, and the lighting being just right and never changing, simply picking up, say, a green apple and placing it in a glass bowl on a kitchen table becomes an all but impossibly difficult problem to solve.” Jeff Bingham – U.S. controls engineer, staff roboticist at Google, as reported September 10, 2024, on WIRED by Hans Peter Brøndmo - Norwegian/U.S. computer scientist, entrepreneur, former VP of Google X.

“The reason it has taken so long for the robotics industry to move forward is because people keep trying to make something that is cool but difficult to achieve rather than trying to find solutions to actual human problems. Technology can be extremely expensive if you don't focus. … The answer is navigation, manipulation, and implementation of more sophisticated intelligence. The idea that a robot will become more aware of its environment, that telling it to ‘go to the kitchen’ means something - navigation and understanding of the environment is a robot problem. Those are the technological frontiers of the robotics industry.”  Colin Angle – U.S. business entrepreneur, co-founder and CEO of iRobot .

“The first rule of any technology used in a business is that automation applied to an efficient operation will magnify the efficiency. The second is that automation applied to an inefficient operation will magnify the inefficiency.” Bill Gates – U.S. business entrepreneur, co-founder and former CEO of Microsoft .

“(T)here are two primary approaches to applying AI in robotics. The first is a hybrid approach. Different parts of the system are powered by AI and then stitched together with traditional programming. With this approach the vision subsystem may use AI to recognize and categorize the world it sees. Once it creates a list of the objects it sees, the robot program receives this list and acts on it using heuristics implemented in code. If the program is written to pick that apple off a table, the apple will be detected by the AI-powered vision system, and the program would then pick out a certain object of 'type: apple' from the list and then reach to pick it up using traditional robot control software. The other approach, end-to-end learning, or e2e, attempts to learn entire tasks like 'picking up an object,' or even more comprehensive efforts like 'tidying up a table.' The learning happens by exposing the robots to large amounts of training data—in much the way a human might learn to perform a physical task. If you ask a young child to pick up a cup, they may, depending on how young they are, still need to learn what a cup is, that a cup might contain liquid, and then, when playing with the cup, repeatedly knock it over, or at least spill a lot of milk. But with demonstrations, imitating others, and lots of playful practice, they’ll learn to do it—and eventually not even have to think about the steps. … But teaching robots to autonomously live and work alongside us is a comparably huge data problem. In spite of simulations and other ways to create training data, it is highly unlikely that robots will “wake up” highly capable one day, with a foundation model that controls the whole system. The verdict is still out on how complex the tasks will be that we can teach a robot to perform with AI alone. From a September 10, 2024, posting on WIRED by Hans Peter Brøndmo – Norwegian/U.S. computer scientist, entrepreneur, former VP of Google X.

“The robot is shaped like a human, but it sure doesn’t move like one. It starts supine on the floor, pancake-flat. Then, in a display of superhuman joint mobility, its legs curl upward from the knees, sort of like a scorpion tail, until its feet settle firmly on the floor beside its hips. From there, it stands up, a swiveling mass of silver limbs. The robot’s ring-light head turns a full 180 degrees to face the camera, as though possessed. Then it lurches forward at you. The scene plays out like one of those moments in a sci-fi movie when the heroes think for sure the all-powerful villain must be done for, but somehow he comes back stronger than ever. Except it’s a real-life video released last month (April, 2024) by the robotics company Boston Dynamics to introduce its new Atlas robot. The humanoid machine, according to the video’s caption, is intended to further the company’s “commitment to delivering the most capable, useful mobile robots solving the toughest challenges in industry today.” Except it’s a real-life video released last month by the robotics company Boston Dynamics to introduce its new Atlas robot. … (R)obots have lagged behind chatbots and other kinds of generative AI because ‘the physical world is extremely complicated, far more so than language,’ (per Marc Raibert, the founder of Boston Dynamics). The company posted its first video of Atlas doing a backflip in 2017; more than six years later, the robot still is not commercially available. ‘The athletic part of robotics is really doing well,’ Raibert told Wired in January (2024), ‘but we need the cognitive part.’ The actual business of Boston Dynamics is comparatively mundane. Currently, its humanoid robots are purely for research and development. Its commercial products—a large robotic arm and a small robotic dog—are used mainly for moving boxes and workplace safety and inspections. … (Boston Dynamics is poised to benefit from new Artificial Intelligence and Machine Learning advancements.) Hardly a week after the launch of ChatGPT in late November 2022, the company announced the creation of a new AI Institute. (In April, 2024), it posted a video about using simulations and machine learning to teach its robot dogs how to move through a range of real-world environments. And the press release for the new Atlas robot explicitly talked up the company’s progress in AI and machine learning over the past couple of years: ‘We have equipped our robots with new AI and machine learning tools, like reinforcement learning and computer vision, to ensure they can operate and adapt efficiently to complex real-world situations.’” From an article posted May 1, 2024, on The Atlantic written by Jacob Stern – U.S. journalist.

“(Amazon) has acquired a team to give robots greater intelligence and dexterity—potentially automating much more of its warehouse operations. Last week (week of August 26, 2024), the ecommerce giant announced (an agreement) to hire the founders of Covariant, a startup that has been testing ways for AI to automate more of the picking and handling of a wide range of physical objects. … Back in the aughts, Kiva (a robotics startup acquired by Amazon in 2012) developed a way to move products through warehouses by having squat robots lift and carry stocked shelves over to human pickers—a trick that meant workers no longer needed to walk miles every day to find different items. Kiva’s mobile bots were similar to those employed in manufacturing, and the company used clever algorithms to coordinate the movement of thousands of bots in the same physical space. Amazon’s mobile robot army grew from around 10,000 in 2013 to 750,000 by 2023, and the sheer scale of the company’s operations meant that it could deliver millions of items faster and cheaper than anyone else. As WIRED revealed last year (in 2023), Amazon has in recent years developed new robotic systems that rely on machine learning to do things like perceive, grab, and sort packed boxes. Again, Amazon is leveraging scale to its advantage, with the training data being gathered as items flow through its facilities helping to improve the performance of different algorithms. The effort has already led to further automation of the work that had previously been done by human workers at some fulfillment centers. The one chore that remains stubbornly difficult to mechanize, however, is the physical grasping of products. It requires adaptability to account for things like friction and slippage, and robots will inevitably be confronted with unfamiliar and awkward items among Amazon’s vast inventory. Covariant has spent the past few years developing AI algorithms with a more general ability to handle a range of items more reliably. … In March, 2024, Covariant demonstrated a chat interface for its robot and said it had developed a foundation model for robotic grasping, meaning an algorithm designed to become more capable as it is fed more data. …  But when WIRED visited one of Amazon’s more automated facilities last year (2023), humans were still required for picking and packing—as well as for helping the odd lost or confused robot. The Amazon–Covariant deal could well see the numbers of humans needed to handle its products dwindle in the decade to come.” Posted to WIRED on September 4, 2024, by Will Knight – U.S. journalist.

“In addition to replacing many jobs, automation will also transform other jobs. Professions involving high touch, personal relationships - such as clergy, dentists, and financial advisors, for instance - face the least risk of automation but will nevertheless be profoundly transformed. … Rather than wringing our hands about robots taking over the world, smart organizations will embrace strategic automation use cases. Strategic decisions will be based on how the technology will free up time to do the types of tasks that humans are uniquely positioned to perform.” Clara Shih – Hong Kong, China born/U.S. business entrepreneur, co-founder and former CEO of Hearsay Social, currently CEO of Salesforce .

"The real opportunity with digital transformation and automation is to reinvent jobs, to have them be more human, to work alongside machines." Erik Brynjolfsson – U.S. academic, author, primary focus on effects of information technologies on business strategy, productivity and performance.

“Automation is not the future of work, but it is the present and a critical component of staying competitive."  Josh Bersin – U.S. research analyst, specializing in areas of human resources, technology, learning, and leadership.

"Automation is not a one-time event, but a continuous process of improvement and innovation."  Anand Deshpande – Indian business entrepreneur, founder and chairman of Persistent Systems.

“Driven by the convergence of enabling technologies such as artificial intelligence, sensor technology, and computer vision … robots are poised to impact how we live and work. … The structured, relatively static nature of industrial manufacturing environments is ideal for robots, which thrive at carrying out predictable and repetitive tasks.  In the automotive industry, robots assemble components, install parts, and handle materials. Robots with cutting, drilling, and milling tools have demonstrated increased quality, reduced costs, and accelerated production. A recent study found that robots can reduce conversion costs by as much as 15% and, when combined with other technologies, process enhancements, and structural layout changes, can yield savings of up to 40%. - The widespread use of robotics in industrial manufacturing and associated economic value have been a driving force for adoption and innovation. With advances in perception and spatial understanding, robots have expanded to more challenging environments, such as construction and agriculture. During site layout, robotic total stations augment the human workforce, accurately measuring angles and distances to ensure construction elements are placed as planned. Robots help survey and inspect job sites, handle materials, and perform welding and fabricating tasks for structural steelwork. In agriculture, automated tractors equipped with precision planting technology accurately plant seeds at optimal depths and spacing so crops grow uniformly. Robots use computer vision and machine learning to identify and remove weeds within rows of desirable crops with precision spraying. Autonomous combines and harvesters operate without human intervention, relying on GPS and other sensor data to work efficiently, optimize paths, and minimize loss by harvesting crops without damaging them. … Collaborative robots, commonly called cobots, offer an entry point into the world of automation. Cobots work alongside humans and are designed to be user-friendly.  From a safety standpoint, cobots have advanced sensors that detect and respond to humans. They alleviate fears by helping people experience the benefits of robotics without losing direct control of the work being performed. Cobots can perform various tasks, from simple pick-and-place operations to more complex processes.  In healthcare settings, cobots can help lift patients, reducing the physical strain on healthcare workers. In retail, they help manage inventory, scanning shelves to track stock levels or assisting with restocking products.  … Robots are often classified based on their application, locomotion, autonomy, or operating environment. While robots come in many forms, technological advancements coupled with the need to adapt machines to the physical environment have led to the development of humanoid robots designed to resemble and mimic humans. …The World Economic Forum estimates that automation could displace 85 million jobs by 2025, but is also predicted to create 97 million new ones.  While there’s no doubt that automation will impact the workforce, researchers and industry leaders are working to discover how. A recent study led by OpenAI’s CEO, Sam Altman, explored the intriguing idea of using government-funded cash aid as a starting point for helping workers who lose income to technological advances. …  Regulatory and ethical considerations will become increasingly important, requiring collaboration among industry stakeholders, regulatory bodies, and policymakers to address data privacy, cybersecurity, and the impact of automation on the workforce.” Posted September 19, 2024 on Fast Company by Aviad Almagor – U.S. technologist, vice president -technology innovation at Trimble Inc.

“As technology for AI, vision systems, autonomous vehicles and more progresses, automation systems can use these tools to take on more complex tasks. Here are four recent technology trends, and how they apply to automation: AI. AI is having a moment in all industries, and automation is no different. Earlier this year, Teradyne Robotics, parent company of cobot company Universal Robots, announced a partnership with Nvidia. UR has worked with the chipmaker to incorporate Nvidia’s CuMotion package into cobot path planning, which can speed the process by as much as 80%. The cobot maker is also collaborating with Nvidia to incorporate AI tools into its PolyScope X operating software. Vision systems. With the addition of 3D vision systems, robots can perform more complex tasks, such as bin picking, material handling and inspection, particularly when parts vary in shape or orientation. MetalQuest automated a bin-picking task with help from a 3D vision system, while Comau has developed a system that uses 2D cameras to create 3D images. AI can advance machine vision even further — Siemens’ Robot Pick AI enables a robot to pick and place unknown objects without training. Easy-to-program robots. Robotic systems for machine-tending are becoming even easier to implement, with users needing to provide only a few pieces of information in many cases. Robots are also becoming even easier to program in applications beyond machine tending. At Athena Manufacturing, welders are responsible for programming welding cobots, and FANUC has released a cobot designed for painting applications. Autonomous robots. Technology has progressed from robots fixed in one place to cobots that can be moved from machine to machine. Now, robots are going fully mobile. Automated guided vehicles (AGVs) and autonomous mobile robots (AMRs) are becoming commonplace in warehouses and beginning to make their way into manufacturing facilities for tasks such as material handling. Boston Dynamics, well-known for its autonomous dog-like inspection robot Spot, has also launched an autonomous robot for warehouses called Stretch. Posted September 18, 2024, on Modern Machine Shop by Julia Hider – U.S. journalist, robots and auonomy correspondent for Gardner Business Media, Inc.

"The key to successful automation is identifying the right processes to automate and ensuring that the automation aligns with your organization's strategic goals."  Tom Wilde – U.S. technologist, CEO at Indico Data .

"The goal of automation is not to replace humans but to augment their capabilities and allow them to focus on tasks that require creativity, critical thinking, and emotional intelligence." Eric S. Yuan – Chinese/U.S. business entrepreneur, chief executive officer and founder of Zoom Video Communications.

“With automation, of course, comes the risk of disappearing jobs. Kernel (a vegan takeout place where the food is made largely by robots created by Steve Ells, the original founder of Chipolte) — and other restaurants are experimenting with robots not only in pursuit of efficiency, but because the industry is facing a chronic labor shortage. The low pay doesn’t help, and the jobs are also exhausting as well as, at times, hazardous. Deep-frying, for instance, is extremely dangerous, which is why one of the most popular cooking robots in the industry simply runs the fry station. … Faster and more automated cooking technology may well be imminent, but humans will still be involved for years to come. Automated pizzaiolos, line cooks, and salad tossers have failed; successful robots typically target a specific task, such as plunging fries into boiling oil.” Posted on The Atlantic on April 18, 2024 by Matteo Wong – U.S. journalist.

“It's robots' increasing capacity to mimic organisms physically and cognitively that has many people scared. Although most Americans may not be apocalyptic worrywarts, they have grave reservations about automation. They're more disposed toward driverless vehicles and robot caregivers, but there's a limit how much innovation they're willing to embrace.” Les Horvitz – U.S. writer.

“Regulation must push companies toward safe innovation and innovation in safety. We are not there yet.… Any regulation of industrial robots stems from existing industrial regulation, which has been evolving for many decades. The Occupational Safety and Health Act of 1970 established safety standards for machinery, and the Robotic Industries Association, now merged into the Association for Advancing Automation, has been instrumental in developing and updating specific robot-safety standards since its founding in 1974. Those standards, with obscure names such as R15.06 and ISO 10218, emphasize inherent safe design, protective measures, and rigorous risk assessments for industrial robots. … As we move into a future where robots are becoming integral to our lives, we can’t forget that safety is a crucial part of innovation. True technological progress comes from applying comprehensive safety standards across technologies, even in the realm of the most futuristic and captivating robotic visions. By learning lessons from past fatalities, we can enhance safety protocols, rectify design flaws, and prevent further unnecessary loss of life.” Posted on The Atlantic on September 6, 2023, by Bruce Schneier – U.S. technologist, fellow and lecturer at the Harvard Kennedy School Executive Education , and Davi Ottenheimer – U.S. technologist, vice president of trust and digital ethics at Inrupt .  

“Technologies that are emerging today will soon be shaping the world tomorrow and well into the future – with impacts to economies and to society at large. Now that we are well into the Fourth Industrial Revolution, it’s critical that we discuss and ensure that humanity is served by these new innovations so that we can continue to prosper.” Mariette DiChristina – U.S. journalist, academic, dean of the College of Communication at Boston University , was the editor-in-chief of the magazine Scientific American from 2009 thought 2019. 

“It is the fusion of these technologies and their interaction across the physical, digital and biological domains that make the fourth industrial revolution fundamentally different from previous revolutions.” Klaus Schwab - German mechanical engineer, economist, founder of the World Economic Forum .