Cooperative Transportation by Humanoid Robots — Learning to Correct Positioning —
- 格式:pdf
- 大小:3.82 MB
- 文档页数:10
Choice of modeThe choice of mode for long distance travel is heavily dependent on the sensitivity of the traveler with respect to time and cost.By and large,business travel is time sensitive,vacation travel is price sensitive,and travel for personal reasons may be either time sensitive or cost sensitive,or both.长距离旅行方式的选择在很大程度上取决于旅行者对时间和价格反应的灵敏程度。
一般来说,商业性旅行对时间反应更灵敏,节假日旅行对价格反应更灵敏,由于私事(个人原因)旅行则对时间或价格的反应都灵敏。
The basic attributes of each mode are schedule, speed, cost, service offered,and perceptions regarding the service offered.每种方式的特性取决于所提供服务的时间速度、价格、服务质量和给人们的感受。
Schedule and speed prescribe the ability of the mode to serve passengers at the times they want and the speed they require; for example, a same-day round trip from Chicago to New York can be accomplished by air only.时间性和速度性要求旅客所采用的运输方式能提供给乘客想要的速度和时间。
公共交通运输系统外文文献翻译(含:英文原文及中文译文)文献出处:Marinov Kim. The fractal structure of Seoul’s public transportation system[J]. Cities, 2013, 20(1):31-39.英文原文Public transportation systemMarinov KimAbstractTransportation systems provide significantly different services than urban suburbs, which often leads to different assumptions about the user's choice of transportation method. The simulation model mentioned in this paper proposes a policy for evaluating the impact of transport services. The mode of transportation is considered to be public transport, including light rail transit (light rail) and buses, plus private cars. In the three-step traveler behavior simulation model, the concept of generalized transportation cost is used. It proposes various types of transportation, as well as advice on residents’ travel options and the quantification of suburban residential community forms, and uses data derived from a typical corridor in Beijing, China. The simulation results show that lowering fares, increasing the comprehensive capacity of public transport, and penalties for private cars are necessary to improve the efficiency of the system and the attractiveness of the suburbs, especially for those withlow income; without roads Pricing will encourage middle-income residents to shift to private cars. At the same time, high-income earners may leave the suburbs due to road congestion; however, improvements in public transportation can attract more short- and medium-distance travelers, but passengers are not sensitive to travel distance.Key words: generalized cost, public transportation, congestion pricing, transportation service, BeijingSince the 1990s, large and medium-sized cities in China have experienced suburbanization, and the spatial structure of these cities has been gradually formed, relying heavily on the advancement of transportation. Due to population explosions and the transformation of the central cities, many suburban towns have developed into residential areas, and most of these newly developed towns and cities are employed by central cities or nearby industrial areas. For example, more than 80% of the residents of the Longkou suburban community work in Beijing or in central cities in developed regions. Almost half of the Beijing community residents are employed in the CBD of Chaoyang District. These areas pose new challenges to traffic policy makers and urban planners in the planning of transportation systems and the provision of operational efficiency.In many cases, border towns connect central cities or industrial parks via highways and city tracks. Compared with traditional cities, suburbanexhibitions exhibit more stable modes of transport use, rely more on public transportation, private cars, and less use of motorized modes (bicycles, walking). Working distance has a greater impact on people's mode of transportation than any single factor. This feature can profoundly affect the suburbanization of the population and restrict some people from immigrating to the suburbs.This article uses Beijing in the northwestern region as a test case to analyze the policies for transport services, mainly the costs and service quality, and will affect the overall transport system and urban spatial structure. The organization of this article is as follows: Section 2 briefly reviews some of the recent literature on the choice of transport mode and compares it with the Haicheng Corridor case in Haicheng City, especially China. The third section discusses the concept of transport costs in general and establishes new concepts, including the use of generalized costs and the obstacles to shifting the cost budget. In the fourth quarter, the passenger transport mode selection behavior simulation model was introduced between public transport and driving, and then it turned to the China experience transport service policy and the selection of Haicheng Haicheng travel mode. In the fifth section, the special mention was made of the rapid suburbanization of the northwestern region using Beijing's light rail and highway. In section VI, the main findings and policy implications are drawn. A large number of research institutions arestudying the choice of transportation mode and individual travel mode. In general, there are three common ways to determine this problem. The first part focuses on the characteristics of each model that influences choice decisions, and the empirical research used to change the results, study people and tourism purposes. For example, the travel-to-work behavior in Accra (Ghana) is primarily determined by the perceived quality of service, commercial commuter cars, and the personal circumstances of the employees, rather than by waiting for the time or the car. In the United Kingdom, to determine the itinerary for visiting relatives and friends, economic factors explain the choice of mode to a large extent, and the reasons for qualitative use of private transport are often secondary (Cohen, Harris, 1998). Examples of images used by Johansson and other Swedish workers indicate that the two sensations of flexibility and comfort affect the individual's choice pattern. According to the experimental field research and statistical analysis in Frankfurt, Germany, there are four types of people. One group seldom pays attention to money and time. The second group attaches great importance to money but does not attach much importance to time. The third group attaches great importance to it. The fourth group seldom pays attention to money but attaches great importance to time. Lin Tancu et al. selected data from 1998 that the Dutch national tourism survey team confirmed spatial allocation, land use and transport infrastructure, and had a major impact on long-distancetravel patterns such as commuting, business and leisure travel.The concept of generalized transportation costs is often used to evaluate and explain tourism behavior. Generalized travel costs include travel time, overtime, money costs, parking fees, and some negligible tips. However, this concept is different in different literature because of different research purposes. For example, generalized cycling costs are as follows: travel time, physical needs, comfort, traffic safety, the risk of bicycle theft, the cost of parked bicycles, maintenance costs, and personal safety. The generalized concept of monetization is usually the conversion of travel time into monetary expenses, increased fees, and fees collected. In this study, the commuter travel behavior was simulated. The three transports are all related to the concept cost. The first one can be defined as budgetary obstacles, including operating expenses, road maintenance fees, and parking fees. The second is the generalized cost and time cost of monetization, including operating expenses, as well as parking fees. Users are presumed to minimize the monetization costs of individuals and differentiate their traveler from travel time and cost. The generalized monetization cost of public transportation is to calculate the time of access to the train station, waiting time, which is calculated based on the progress, waiting time, and fare. Monetization costs include travel time, fees, car operating costs, and parking fees at work locations. The third concept is to determine the cost of the mode transfer cost, that is,congestion caused by discomfort. It is speculated that if the passengers in the vehicle have far exceeded capacity, new passengers will be transferred to other affordable transport methods.Roads that run through cities and villages are usually not only private cars but also public cars. From previous experience, road pricing can greatly increase the user's remaining area, and the cost of public transportation can greatly reduce the congestion caused by long-term travel, especially if the user largely loves public transportation. Option 2 shows that high-income travelers have to abandon plans for regular migration to the suburbs, mainly due to the serious congestion caused by free use. The other group is passengers who have deeply affected middle-income people. They switched from public transport to private car driving.The reduction in bus fares and the increase in bus lines have led many middle-income residents to move to the bus, while car users have shifted to the light rail, especially those short-distance travel. In addition, the study found that a comprehensive policy to improve public transport services and some private car punitive measures will help improve system efficiency and the attractiveness of suburban communities.This integrated system includes the reduction of light rail and bus fares, the improvement of service quality, and the collection of highway tolls. It should be pointed out that the light rail has been designed toincrease the use of public handrails to make public transport more attractive to users, and thus reduce road congestion. According to the simulation results, the reduction in LRT and bus fares and the increase in passenger capacity make this model more attractive to those low- and middle-income travelers. The end result is that more and more people migrate to the suburbs, from low-income to high-income classes. Of course, improvement of public transport services requires government subsidies. This is also a worldwide phenomenon. The economic travel distance of each model is hardly affected by the policies of different transport services. Public buses are used for short-distance travel, while middle- and long-distance people prefer light rail. On the other hand, car users do not matter. The results show that these low-income citizens have moved to suburban economic housing due to existing transport service policies, including relatively high fees for public transportation and roads, and limited light rail and bus lines. Of course, all modes of transport have a certain degree of influence on people working from home.Urban transport is a major area of government policy throughout the world. Transport policy will also affect the urban form, especially suburbanization. In Beijing, many economic apartments are designed for the construction of low-income residents. There are two main issues before the decision on major issues. First, from past experience, a large number of permanent residents in suburban communities are middle- orhigher-income people. And many owners still live in the city center, and suburban houses are only used for vacations. In addition, residents are constantly complaining about traffic congestion and relatively high toll roads, especially during peak hours. The simulation results of this study explain these phenomena and try to give corresponding policy implications.中文译文公共交通运输系统Marinov Kim摘要运输系统提供了与城市郊区显着不同的服务,这通常会导致有关用户选择交通方式的不同假设。
Unite One Transportation SystemText A The role of transportationTransportation is foundational to the development and operation of a modern society. It permits the specialization of work effort necessary to achieve efficiency and productivity. Geographically distant resources become accessible with transportation. The economic growth of any society is directly related to the availability of transportation.交通运输作为现代社会发展和运行的基础实现了分工的专业化,从而提高了工作效率和生产率。
通过交通运输,在地理位置上显得遥不可及的地理资源也变得唾手可得。
在任何社会,其经济的增长与交通便利性密不可分。
There are many different definitions of transportation in various books and dictionaries. However, the common element among these definitions is movement-the change of the physical location of goods or passengers. Therefore, transportation can be defined as the movement of freight and passengers from one location to another in this context. Products must be moved from the location where they were produced to the location where they are consumed. People must move from the location where they live to the location where they want to take part in a certain activity. We need transportation in order to survive and improve our standard of living. Transportation affects our economic, social, and political development.在不同的书籍和字典中对于交通运输有不同的定义。
交通专业英语作文(中英文版)Title: Transportation Specialist English EssayIn the modern world, transportation plays a crucial role in the development and progress of society.It is an essential aspect of our daily lives, enabling us to travel, transport goods, and connect with one another.The field of transportation is constantly evolving, with new technologies and innovations shaping the way we move people and goods from one place to another.As a transportation specialist, it is important to have a strong understanding of the industry and its terminology in order to effectively communicate and contribute to its growth.The transportation industry encompasses a wide range of modes, including road, rail, air, and water.Each mode has its own set of terms and concepts that specialists need to be familiar with.For example, in road transportation, terms such as traffic flow, congestion, and Intelligent Transportation Systems (ITS) are commonly used.In rail transportation, terms like trackage, signaling, and freight transportation are important.In air transportation, specialists need to be familiar with terms such as air traffic control, airport operations, and aviation safety.In addition to understanding the different modes of transportation, transportation specialists also need to be aware of the latest trends anddevelopments in the industry.This includes topics such as sustainable transportation, electric vehicles, autonomous vehicles, and smart cities.Staying informed about these trends is important for developing innovative solutions and improving the efficiency and effectiveness of transportation systems.Furthermore, effective communication is key in the field of transportation.Transportation specialists need to be able to convey complex information clearly and concisely to a variety of audiences, including policymakers, engineers, and the general public.This requires strong writing and speaking skills, as well as an understanding of industry-specific terminology.In conclusion, as a transportation specialist, it is important to have a comprehensive understanding of the industry and its terminology.This includes being familiar with the different modes of transportation, staying informed about the latest trends and developments, and having strong communication skills.By mastering these skills, transportation specialists can contribute to the growth and development of the industry, and help create efficient and effective transportation systems for the future.。
Transportation plays a pivotal role in the modern world,connecting people,goods, and services across vast distances.Heres a detailed essay on the topic of transportation:Title:The Evolution and Impact of TransportationIntroduction:The history of transportation is as old as civilization itself.From the invention of the wheel to the development of modern vehicles,the way we move has transformed dramatically.This essay will explore the evolution of transportation,its current state,and its impact on society and the environment.Body:1.Historical Perspective:In ancient times,transportation was primarily by foot or by animal,with the wheel revolutionizing the way goods were moved.The introduction of the sailboat and later,the steam engine,allowed for faster and more efficient travel over water and land.2.Development of Modern Transportation:The19th and20th centuries saw the rise of the automobile,airplane,and train,which significantly reduced travel times and expanded the reach of commerce and exploration. The mass production of cars and the construction of highways and airports have made personal and commercial transportation more accessible and faster.3.Types of Transportation:Road:Cars,buses,and motorcycles are common for personal and public transportation. Rail:Trains offer a more efficient mode of transportation for long distances and heavy cargo.Air:Airplanes provide the fastest means of longdistance travel,connecting continents and countries.Sea:Ships and boats are crucial for international trade and the movement of large quantities of goods.4.Impact on Society:Transportation has brought people closer together,fostering cultural exchange and economic growth.It has also enabled the rapid movement of goods,contributing to the global economy and the availability of diverse products.5.Environmental Considerations:The rise in transportation has led to increased carbon emissions,contributing to climate change.There is a growing need for sustainable transportation options,such as electric vehicles and public transit systems,to reduce the environmental impact.6.Future of Transportation:Innovations like selfdriving cars,highspeed trains,and electric aircraft are on the horizon,promising even greater efficiency and reduced environmental impact.The integration of technology,such as GPS and realtime traffic updates,is making transportation more efficient and convenient.Conclusion:Transportation is a fundamental aspect of modern life,shaping our ability to connect, trade,and explore.While it has brought about significant benefits,it also presents challenges that must be addressed through sustainable practices and technological advancements.The future of transportation will likely be defined by its ability to balance efficiency,accessibility,and environmental responsibility.。
U n i t T w o T r a n s p o r t a t i o n M o d e s Text A Major transportation modesThe evolution of transportation has been closely linked to the development of humankind throughout the history. Transportation’s early function was to meet the basic need of hauling good supplies. But with the formation of tribes, then peoples, and finally nations, the societal and economic functions of transportation became more and more complex. At first there was mobility required for individuals, households, and animals to protect them against, and to escape from, the dangers of natural disasters and tribal aggressions, and in search for the best place to settle. As tribal groups formed and gradually established their geographical identity, transportation was increasingly needed to open up regions for development, to provide access to natural resource, to promote inter-communal trade, and to mobilize territorial defense.交通运输的演化与人类的整个发展历史密不可分。
交通运输英语作文In the realm of human progress, the development of transportation has been a pivotal element that has shaped the course of history. From the invention of the wheel to the modern-day bullet trains and autonomous vehicles, transportation has evolved dramatically, and its future promises to be even more exciting.The early days of transportation were marked by the reliance on human and animal power. The wheel, one of the most significant inventions, revolutionized the way goods and people were moved from one place to another. With the advent of the Industrial Revolution, steam power and later internal combustion engines transformed transportation, leading to the creation of trains, automobiles, and airplanes.In the modern era, transportation has become faster, safer, and more efficient. High-speed rail networks like those in Japan and China have redefined long-distance travel, while advancements in aviation have made it possible to traverse the globe in a matter of hours. The rise of the internet and digital technology has also had a profound impact on transportation systems, with real-time tracking and smart routing becoming the norm.Looking ahead, the future of transportation is set to be defined by innovation and sustainability. Electric vehicles are becoming increasingly popular as a means to reduce carbonemissions and combat climate change. Moreover, the concept of autonomous vehicles is no longer a figment of science fiction, with companies like Tesla and Waymo leading the charge inself-driving technology.The integration of artificial intelligence and machinelearning is expected to further enhance transportation systems. Smart cities with interconnected transportation networks will optimize routes, reduce congestion, and improve overall efficiency. Additionally, the exploration ofalternative modes of transportation, such as hyperloopsystems and even space travel, is pushing the boundaries of what is possible.In conclusion, the evolution of transportation is a testament to human ingenuity and our unrelenting pursuit of progress.As we continue to innovate and adapt to the challenges of the future, the horizon of transportation holds the promise of a more connected, efficient, and environmentally friendly world.。
railway coaches铁路客车soft berth tickets软卧票lower berth下铺sea sickness晕船motion sickness晕车express train特快列车trolley电车platform站台luggage rack行李架railway station火车站subway/underground railway地铁cross junction交叉口traffic lights交通灯passenger train旅客列车ordinary road普通公路traffic jam交通拥挤boat time-table船期表book预定home port母港passenger ferry客渡船charging station充电站heavy traffic交通高峰dispatch调度mass transportation公共交通运输economy class经济舱first class头等舱商务舱business classairport terminal机场候机楼magnetic levitation train/maglev磁悬浮列车maiden flight首飞flight crew机组人员cabin baggage机舱行李ticket office售票处traffic flow交通流distant signal预告信号cab signal机车信号running signal行车信号shunting signal调车信号shunting neck牵出线parking and riding停车换乘CY=cylinder圆筒汽缸bogie/truck转向架GWT=gross weight毛重总重shipper托运人carrier承运人container集装箱less-than-truck load卡车零担urban freight systems城市货运系统urban freight network城市货运网络urban transportation systems城市交通系统traffic congestion交通拥挤intelligent transportation systems智能运输系统global positioning systems全球定位系统bids of lading提单electronic vehicle tagging汽车电子标签jam拥挤,堵塞headway车头(间)时距platoon车队regular transit bus常规公交traffic scenarios交通环境backing shunting推进调车fly shunting溜放调车jump shunting驼峰调车interlocking联锁的moving block移动闭塞block section闭塞分区catenary接触网traffic control调度dispatcher/traffic controller调度员slip事故block闭塞motorway/autoroute/expressway高速公路robustness健壮性、鲁棒性Intelligent Transport ManagementSystems(ITMS)智能交通管理系统red traffic light infringement闯红灯license plate车牌Intelligent Transportationsystems(ITS)智能交通系统electronic tags电子标签heavy haul重载运输passing/crossing station会让站overtaking station越行站intermediate station中间站district station区段站marshalling station编组站marshaling yard(集装箱)编组场technical station技术站passenger station客运站freight station货运站section区间district区段box car箱式车major trunk line干线seasonal demand季节性需求speed up/speed increase提速double-track percentage复线率electrified percentage电化率average travel time平均旅行时间CFS-container freight station集装箱货运站combined transportation联合运输through transportation直达运输transfer transportation中转运输drop and pull transportation甩挂运输container transportation集装箱运输door to door门到门full container load整装箱less than container load拼装箱maintenance time天窗时间driver’s license驾驶执照marine Terminal港口站airport Terminal航站楼bus Terminal公交终点站ferry Terminal轮渡站terminal3T3航站楼hub枢纽central station中央火车站Junction中间站-过轨站marine Terminal港口站airport Terminal航站楼bus Terminal公交终点站ferry Terminal轮渡站terminal3T3航站楼hub枢纽central station中央火车站铁路市场份额rail market share首航maiden flight添乘On-board inspection on traincrew’s word by management/on traincrew’s work再生制动regenerative brakeATIS先进的交通信息系统ATMS先进的交通管理系统AVCS先进的车辆控制系统AGV自动导引小车BRT城市快速公交系统CTC调度集中ETC电子收费EMU电力动车组ECR有效客户反应EOQ经济订货批量EOS电子订货系统HRT重轨交通HY驼峰编组场驼峰调车厂ITS智能交通系统LCL零担货FCL整箱货LRT轻轨交通UT城市交通单元列车UR地下铁路MY编组场YR厂修MRP物料需求计划MRPⅡ制造资源计划APTS先进的公共交通系统EOT列尾检查装置CRH和谐号动车组/中国高速铁路ATC(automatic train control system)列车自动控制系统The vehicle is actually easier to control than normal cars because you don’t have to change gears while driving.电动汽车实际上比普通汽车更容易控制,因为驾驶过程中你不需要换挡。
TransportationTransportation plays a crucial role in various aspects of modern society and the economy. Its importance can be highlighted in the following ways:Economic Growth: Transportation is a driver of economic growth. It enables the movement of goods and services, facilitating trade and commerce. Efficient transportation systems lower production costs, increase market access, and boost overall economic productivity.Employment: The transportation sector itself is a significant source of employment. It includes jobs in areas like logistics, trucking, shipping, aviation, and public transportation. Moreover, transportation enables people to access jobs in other industries by providing mobility.Accessibility: Transportation connects people to essential services such as healthcare, education, and public services. It ensures that individuals can reach hospitals, schools, and government offices, improving their quality of life.Globalization: Transportation is fundamental to globalization. It allows for the movement of people and goods across borders, fostering international trade, cultural exchange, and cooperation. Global supply chains rely on efficient transportation networks.Urbanization: As more people move to cities, efficient public transportation becomes crucial to managing urban congestion and reducing pollution. Mass transit systems can alleviate traffic and make cities more livable.Emergency Response: Transportation is vital for disaster relief efforts and emergency response. It allows for the swift movement of personnel, supplies, and equipment to affected areas during natural disasters, accidents, or crises.Tourism: Tourism relies heavily on transportation. Tourists need to reach their destinations, whether by air, sea, or land. Transportation infrastructure and services are essential for the tourism industry's success.Environmental Impact: The transportation sector has a significant impact on the environment due to emissions from vehicles and infrastructure development. Sustainable transportation options, such as electric vehicles and public transit, are critical for reducing these environmental effects.Social Equity: Access to transportation can be a matter of social equity. Lack of access to affordable and efficient transportation can limit opportunities for employment, education, and healthcare, disproportionately affecting marginalized communities.Innovation: Transportation continually evolves with technological advancements. Innovations such as autonomous vehicles, high-speed trains, and electric aircraft are changing the way people and goods are transported, with potential benefits for efficiency and sustainability.In conclusion, transportation is a fundamental pillar of modern society and the global economy. It impacts various aspects of daily life, from economic development to social inclusion, and its efficiency and sustainability are critical for addressing current and future challenges.。
Cooperative Transportation byHumanoid Robots—Learning to Correct Positioning—Y utaka Inoue,Takahiro Tohge,Hitoshi IbaDepartment of Frontier Informatics,Graduate School of Frontier Sciences,The University of Tokyo,7-3-1Hongo,Bunkyo-ku,Tokyo,113-8656,Japaninoue,tohge,iba@miv.t.u-tokyo.ac.jpAbstract.In this paper,we describe a cooperative transportation problem with twohumanoid robots and introduce a machine learning approach to solving the problem.The difficulty of the task lies on the fact that each position shifts with the other’swhile they are moving.Therefore,it is necessary to correct the position in a real-time manner.However,it is difficult to generate such an action in consideration of thephysical formula.We empirically show how successful the humanoid robot HOAP-1’scooperate with each other for the sake of the transportation as a result of Q-learning.1IntroductionIn this paper,wefirst clarify the practical difficulties we face from the cooperative transporta-tion task with two bodies of humanoid robots.Afterwards,we propose a solution to these difficulties and empirically show the effectiveness both by simulation and by real robots.In recent years,many researches have been conducted upon various aspects of humanoid robots[1][2].Since humanoid robots have physical features similar to us,it is very impor-tant to let them behave intelligently like humans.In addition,from the viewpoint of AI or DAI(Distributed AI),it is rewarding to study how cooperatively humanoid robots perform a task just as we humans can.However,there have been very few studies on the cooperative behaviors of multiple humanoid robots.Thus,in this paper,we describe the emergence of the cooperation between humanoid robots so as to achieve the same goal.The target task we have chosen is a cooperative transportation,in which two bodies of humanoids have to cooperate with each other to carry and transport an object to a certain goal position.As for the transportation task,several researches have been reported on the cooperation between a human and a wheel robot[3][4]and the cooperation among multiple wheel robots[5][6].However,in most of these studies,the goal was to let a robot perform a task instead ofa human.Research to realize collaboration with a legged robot includes lifting operations of an object with two robots[7]and box-pushing with two robots[8].However,few studies have addressed cooperative work using similar legged robots.It is presumed that body swinging during walking renders cooperative work by a legged robot difficult.Therefore,it is more(a)Direct transfer(b)Trunk-based transferFigure1:Two kinds of tasks.difficult for a humanoid robot to carry out a transportation task,because it is capable of more complicated motions and less stable than a usual legged robot.In leader-follower type control[9][10],which is often used for cooperative movement,it is essential that a follower robot acquire information such as the position and velocity of an objectfluctuated by the motion of a leader robot.This information is usually obtained by a force sensor or wireless communication.Such a method is considered to be effective for a robot with a stable center of gravity operating with less information for control.However, much information must be processed simultaneously to allow a humanoid robot to perform complicated actions,such as transporting an object cooperatively,with its difficulty to control caused by its unstable body balance.It would be expensive to build a system that carries out optimal operation using this information.One hurdle in the case where multiple humanoid robots move carrying an object coop-eratively is the disorder of cooperative motion by body swinging during walking.Therefore in this paper,learning is carried out to acquire behavior to correct a mutual position shift generated by this disorder of motion.Q-learning is used for this learning.We will show that behavior to correct a position shift can be acquired based on the simulation results of this study.Moreover,according to this result,the applicability to a real robot is investigated.This paper is organized as follows.The next section explains the clarified problem diffi-culties with the cooperative transportation.After that,Section3proposes our method to solve the problem.Section4presents an experimental result in the simulation environment.Then section5shows an experimental result with real robots.Section6discusses these results and future researches.Finally,a conclusion is given in Section7.2Operations and Problem in PracticeWe conducted an experiment assuming tasks to transport a lightweight object all around, aiming to extract specific problems from using two humanoid robots:HOAP-1(manufactured by Fujitsu Automation Limited).Dimensions of a HOAP-1are223x139x483mm(width, depth,and height)with a weight of5.9kg.It has two arm joints with4degrees of freedom each,and two leg joints with6degrees of freedom each:20degrees of freedom in all for right and left.(a)Normal position(b)Horizontal slide(c)Approach(d)Spinning around(e)Normal position(f)Horizontal slide(g)Approach(h)Spinning aroundFigure2:Normal positions and unexpected movements.Transportation tasks are of the following two types:(i)each of the two robots transports an object of150x150x200mm and a weight of80g in the same direction(Figure1(a)), and(ii)each of two robots transports a base of150x150x50mm and a weight of20g,with two objects of120x240x6mm and a weight of8g(Figure1(b))thereon.Actually,it seems to be more practical for two robots to have a single object.However, unless both robots move synchronously in the desirable direction,too much load will be given to the arms of robots,which may often cause the mechanical trouble in the arm and the shoulder.Therefore,for the sake of safety,we divide a transporting object or a stretcher into two parts,each of which a humanoid robot carries and transports synchronously.It is assumed in experiment(i)that the arm movement can cancel the position shift,and that the distance and angle that can be cancelled would be in the space between two objects. On the other hand,experiment(ii)assumes that two robots carry a stretcher with an object on it.A sponge grip is attached on each robot arm,so that an object would not slip off the arm during the experiment.The two robots operate in Master-Slave mode.That is,the Master robot transmits data corresponding to each operation created in advance to the Slave robot;the two robots start a motion in the same direction simultaneously.The created basic motions consist of the fol-lowing12patterns:forward,backward,rightward,leftward,half forward,half backward,half rightward,half leftward,right turn,left turn,pick up,and put down.These basic motions are combined to allow the two robots to transport an object.Each experiment was conducted about10times.The results indicated that unintentional motions such as lateral movement(Figures2(b),2(f))and back-and-forth movement(Fig-ures2(c),2(g))by sliding from the normal position(Figures2(a),2(e)),and rotation(Figures 2(d),2(h))occur frequently in basic transportation motions such as forward,backward,right-ward,and leftward.This is considered mainly to result from swinging during walking and the weight of the object.Such a position shift can be cancelled,if not much,by installing a force sensor on a wrist and moving arms in the load direction.However,a robot’s position must be corrected in caseFigure3:The operation overview.of a shift beyond the limitation of an arm.Improper correction may cause failure of an arm or a shoulder joint and breakage of an object.3Learning Model of a RobotThe practical problem of transporting an object is the possibility that a robot falls during movement,due to loss of body balance in connection with a load on the arm by a mutual position shift after moving.Therefore,it is important to acquire behavior for correcting the position shift generated from movement by learning.A situation is assumed in which two robots move face to face while maintaining the distance within a range to transport an object stably.This motion can be divided into two stages:one in which the two robots move simultaneously,and one in which one robot corrects its position.Simultaneous movement of two robots is controlled by wireless communication.A shift over a certain limit of distance or angle in this motion will be corrected by one robot according to behavior acquired by learning.Figure3shows the motion overview for conducting a transportation task.In thefirst stage,the Master robot performs a motion programmed in advance;simultaneously,it issues directions to perform the same motion to the Slave robot.If there is no position shift after movement,the process forwards to the next stage;otherwise,the position is corrected with the learning system.We have tried to realize a cooperative transportation task by repeating the series of thisflow.Learning for position correction is carried out with Q-learning.Q-learning guarantees that the state transition in the environment of a Markov decision process converges into the optimal direction.However,it requires much time until the optimal behavior obtains a reward in the early stage of learning.Thus,it takes time for the convergence of learning.Furthermore, because all combinations of a state and behavior are evaluated for a predetermined Q value, it is difficult to follow environmental change.A CCD camera is attached to one robot to obtain information required for learning fromthe external environment.The external situation is evaluated with images from this CCDFigure4:Status segmentation.Figure5:Image of selected actions.camera.Based on the partner robot’s position projected on the image acquired by the CCD camera,a state space is arranged as shown in Figure4.It is divided into three states:vertical, horizontal,and angular.Hence,the total number of states of the environment is27.If the vertical,horizontal,and angular positions are all centered,the goal will be attained.We assumed6behaviors which a robot can choose among the12patterns mentioned in Section2.They are the especially important motions of forward,backward,rightward, leftward,right turn,and left turn.Figure5depicts all these motions.A given motion is not necessarily carried out ideally in practice.Moreover,a move error may arise depending on characteristics of each robot.Therefore,it is necessary to conduct learning including such an indefinite component.4Learning in Simulation EnvironmentThe learning model stated in the preceding section has been realized in a simulation envi-ronment.This simulator sets a target position at a place of constant distance from the front of the partner robot,which is present in a plane.A task will be completed if the learning robot reaches the position and faces the partner robot.The target position here ranges in dis-tance movable in one motion.In this experiment,back-and-forth and lateral distances and the(a)Earlier trajectory (b)Acquired trajectoryFigure 6:Learning results with Q-learning.Table 1:Numbers of average movement.Q-learning IterationsFailuer Recovery 1,000times 10,000times 100,000timesHorizontal RL 8.88.07.4side LR 14.611.010.4Approach NF 15.07.4 5.6and away FN 4.84.6 2.4Spinning RLS 21.223.820.2around LRS28.629.624.8rotational angle movable in one motion are assumed to be constant.That is,if the movable distance in one step is about 10cm back-and-forth and 5cm laterally,the range of the target point will be 50cm 2.In this range,the goal will be attained if the learning robot is in place where it can face the partner robot with one rotation motion.The Q-learning parameters for simulation were as follows:the initial Q value,Q 0,was 0.0,the learning rate was 0.01,the reduction ratio was 0.8and the reward was 1.0for the task achievement.Behavior patterns obtained by simulation with the Q-learning approach in the early stage and acquired by learning are shown in Figures 6(a)and 6(b),respectively.In the early stage,motions are observed such as walking to the same place repeatedly and going to a direction different from the target position.Behavior approaching the target position is gradually ob-served as learning progresses;finally,behavior is acquired to move to the target position and turn to the front with relatively few motions.5Experiments with real robotsFollowing the simulation results described in the previous section,we conducted an exper-iment with real robots to confirm their applicability.For the recovery from the horizontal left (right)slide,a humanoid robot was initially shifted leftward (rightward)against the op-ponent robot by 7.5cm.On the other hand,it was initially moved forward (backward)from the correct position by 10cm for the recovery from front (back)position.In case of the ro-tation failure,the robot was shifted either leftward or rightward by 10cm and rotated toward the opponent by 30degrees.The actions used for the recovery were of six kinds,i.e.,halfFigure7:Behavior of LR with short-time learning and full learning.forwardChalf backwardChalf rightwardChalf leftwardCright turn and left turn.In order to simplify the whole operation,the image data for a CCD camera was preprocessed manually to produce a sensor input.For this experiment,robots started from one of the three patterns shown in Figures2(b), (c)and(d),which were classified as the failure of actions(see section2).We employed two HOAP-1’s,one of which used the learning results,i.e.,the acquired Q-table,so as to generate actions for the sake of recovery from the failure.Q-learning was conducted by simulation with different numbers of iterations,i.e.,1,000,10,000,and100,000iterations.The learning parameters were the same as in the previous section.Table1shows the averaged numbers of actions for the sake of recovery from the above three failure patterns.In Table1:RL represents the slide recovery from the right,LR is the slide recovery from the left,NF stands for the distance recovery from the front,FN is defined as the distance recovery from the back,RLS and LRS are respectively the angle recovery from the right and from the left.The averaged numbers of required actions were measured over5runs for each experimental condition,i.e.,with different Q-learning iterations.As can be seen in Figure7,in case of the recovery from”horizontal slide”,the robot often moved forward or made rotations in vain when using the Q-table acquired with1,000 iterations.These actions were observed more often when sliding from left to right,which prevented effective recovery from the failure.With the increase of learning iterations,the required number of actions decreased so as to show the effectiveness of the learning.With1,000iterations,more actions were needed to recover from the front position to theback.This is because the robot had acquired the wrong habit of moving leftward when theFigure8:Behavior of NF with short-time learning and full learning.opponent robot was approaching(see Figure8).This habit has been corrected with10,000 iterations,so that much fewer actions were required for the purpose of repositioning.The recovery from”spinning around”seems to be the most difficult among the three pat-terns.For this task,the movement from the slant to the front(see Figure9)was observed with 10,000iterations,which resulted in the increase of required actions.This action sequence was not observed with1,000iterations.Figures7,8and9shows a typical robot trajectory by means of the Q-table acquired with100,000iterations.As can be seen from thefigure,the ac-tions for the recovery were gradually optimized for each failure pattern.Note that with1,000 iterations,the movement toward the wrong direction,other than the destination,had been ob-served.However,after the long term learning,this behavior disappeared and the robot learned to correct the failure positioning effectively.6DiscussionWe have established a learning system for the cooperative transportation by simulation and confirmed its real-world applicability by means of real robots.The effective actions were acquired for the sake of recovery from the position failure as a result of simulation learning.In a real environment,at the earlier stage of learning,we have often observed the un-expected movement to a wrong direction by real humanoid robots;which was also the case with the simulation.In the middle of learning,the forward movement was more often ob-served from the slant direction.These types of movements,in fact,had resulted in the betterlearning performance by simulation,whereas in a real environment they prevented the robotFigure9:Behavior of LRS with short-time learning and full learning.from moving effectively.This is considered to be the distinction between simulation and a real-world environment.In order to solve the difficulty with the distinction,learning in the real world is essential. For this purpose,we are currently working on the integration of GP and Q-learning in a real robot environment[11].This method does not need a precise simulator,because it is learned with a real robot.In other words,the precision requirement is met only if the task is expressed properly.As a result of this idea,we can greatly reduce the cost to make the simulator highly precise and acquire the optimal program by which a real robot can perform well.We espe-cially showed the effectiveness of this approach with various types of real robots,e.g.SONY AIBO or HOAP-1.In the above experiments,only one humanoid robot tried to make actions for the sake of recovery.However,more practical and effective will be both robots moving simultaneously to recover from the failure.For this purpose,two robots will have to learn to cooperate with each other to generate recovery actions.This task will require a more efficient learning scheme. We are in pursuit of this topic by using Classifier System[12].7ConclusionSpecific problems were extracted in an experiment using a practical system in an attempt to transport an object cooperatively with two humanoid robots.The result proved that both body swinging during movement and the shift in the center of gravity,by transporting an object,caused a shift in the position after movement.Therefore,the learning method for correcting aposition shift in a transportation task was modeled.The learning of behavior was performed in a simulation environment using Q-learning.Moreover,the applicability to a real robot has been verified by using the obtained results.Learning in practice will be carried out in the future based on behavior learned in the simulation.Thereby,shortening of the learning time in practice is anticipated.Reduction of learning trials in a practical environment is an important subject because one learning elessonf requires much time in practice.In addition,although only one robot corrects the shift at movement,it is more efficient that the partner robot also corrects its position to reach the target point.Therefore,study is in progress on a method whereby one robot,the one that learns about position correction by carrying a CCD camera,issues directions of motion to the partner robot according to the situation.By resolving these subjects,the authors wish to realize efficient cooperative transportation to arbitrary places.References[1]K.Y okoi,et al.,“Humanoid Robot’s Application in HRP”,In Proc.of IARP International Workshop onHumanoid and Human Friendly Robotics,pp.134-141,2002.[2]H.Inoue,et al.,“Humanoid Robotics Project of MITI”,The1st IEEE-RAS International Conference onHumanoid Robots,Boston,2000.[3]O.M.AI-Jarrah and Y.F.Zheng,“Armmanipulator Coordination for Load Sharing using V ariable Compli-ance Control”,In Proc.of the1997IEEE International Conference on Robotics and Automation,pp.895-900,1997.[4]M.M.Rahman,R.Ikeura and K.Mizutani,“Investigating the Impedance Characteristics of Human Armfor Development of Robots to Cooperate with Human Operators”,In CD-ROM of the1999IEEE Interna-tional Conference on Systems,Man and Cybernetics,pp.676-681,1999.[5]N.Miyata,J.Ota,Y.Aiyama,J.Sasaki and T.Arai,“Cooperative Transport System with Regrasping Car-like Mobile Robots”,In Proc.of the1997IEEE/RSJ International Conference on Intelligent Robots and Systems,pp.1754-1761,1997.[6]H.Osumi,H.Nojiri,Y.Kuribayashi and T.Okazaki,“Cooperative Control of Three Mobile Robots forTransporting A Large Object”,In Proc.of International Conference on Machine Automation(ICMA2000), pp.421-426,2000.[7]M.J.Matari´c,M.Nillson and K.T.Simsarian,“Cooperative Multi-robot Box-pushing”,In Proc.of the1995IEEE/RSJ International Conference on Intelligent Robots and Systems,pp.556-561,1995.[8]H.Kimura and G.Kajiura,“Motion Recognition Based Cooperation between Human Operating Robot andAutonomous Assistant Robot”,In Proc.of the1997IEEE International Conference on Robotics and Automation,pp.297-302,1997.[9]J.Ota,Y.Buei,T.Arai,H.Osumi and K.Suyama,“Transferring Control by Cooperation of Two MobileRobots”,The Journal of the Robotics Society of Japan(JRSJ),vol.14,no.2,pp.263-270,1996.[10]K.Kosuge,T.Osumi and H.Seki,“Decentralized Control of Multiple Manipulators Handling an Object”,In Proc.of the1996IEEE/RSJ International Conference on Intelligent Robots and Systems,pp.318-323, 1996.[11]S.Kamio,H.Mitsuhashi and H.Iba,“Integration of Genetic Programming and Reinforcement Learningfor Real Robots”,In Proc.of the Genetic Computation Conference(GECCO2003),2003.[12]L.B.Booker,D.E.Goldberg,and J.H.Holland,“Classifier Systems and Genetic Algorithms”,In MachineLearning:Paradigms and Methods,MIT Pres,1990.。