DTEVV Workshop
February 4, 2026
California, MD

Register Now
  • Home
  • Events
  • The Development, Test, Evaluation, Verification & Validation (DTEVV) Workshop

The Development, Test, Evaluation, Verification & Validation (DTEVV) Workshop

Pilot flies drone in open lab spaceThe DTEVV Workshop will be held February 4, 2026 at the USMSM SMART Building.

This event will map out the priority research and workforce challenges that need to be addressed to advance the DTEVV of autonomous technologies.

You must be registered to attend. Registration closes January 27, 2026.

REGISTER NOW

Technology continues to rapidly evolve, including the prominent integration of intelligent/autonomous capabilities. Systems - including those intended for the military, homeland security, and first response domains - are moving closer to greater intelligence every day. Every emergent capability and system require some form of assessment; current test and evaluation practices are unable to keep pace with this evolution. The result is a slowdown in the development, assessment, certification, and adoption of these technologies.

The Development, Test, Evaluation, Verification & Validation (DTEVV) Workshop will bring together T&E and autonomous systems professionals from the United States government, academia, and industry to map out the priority research and workforce challenges that need to be addressed to advance the DTEVV of these technologies.

As a workshop participant, you will:

  • Highlight existing and projected research priorities in the autonomous system landscape that will require T&E/V&V
  • Hear lessons learned and best practices from other DTEVV practitioners
  • Provide input toward a DTEVV of Autonomy curriculum that will shape the future workforce
  • Contribute your T&E/V&V of autonomous systems 'wants' and 'needs' to the conversation to help identify priority topic areas

You must be registered to attend. Registration closes January 27, 2026. Register Now

Please note: Programming is still in flux. Keep an eye on this page for the most current updates to the schedule and speaker list.

This workshop is being hosted by the University of Maryland MATRIX Lab in partnership with the University of Maryland Applied Research Laboratory for Intelligence and Security (ARLIS) with support from the Office of Naval Research, the University of Maryland Clark School of Engineering, and the University of Maryland Institute for Systems Research. Email matrix@umd.edu if you have any questions about this event.

 

8:00 a.m. Registration (Coffee and light refreshments will be available) -----
8:30 a.m. Welcome Reza Ghodssi
Executive Director of Research and Innovation
Distinguished University Professor
University of Maryland MATRIX Lab
8:35 a.m. Opening Remarks Samuel Graham, Jr.
Dean, A. James Clark School of Engineering
University of Maryland
8:45 a.m. Workshop Overview Donald "Bucket" Costello III
Director of Test and Evaluation of Autonomous Systems
University of Maryland MATRIX Lab
9:00 a.m.

AM Keynote: From "Cool" to Confident: The Unavoidable Journey of Robot Testing and Evaluation

Robot performance evaluation is often treated as an afterthought when we are heads down, working on something cool, and rightly so. Methodical performance evaluation can sometimes slow down this process of discovery. Nevertheless, we eventually have to test (formally or informally) whether the thing we conjured up can accomplish the task it was created to complete. This talk will present a brief history of the development of performance metrics, test methods, artifacts, and standards for robots and autonomous systems (and their related technologies) at the National Institute of Standards and Technology (NIST), with examples of and lessons learned from both successes and failures. Full abstract

 Kamel "Kam" Saidi
Program Manger and Group Leader, Intelligent Systems Division (ISD)
National Institute of Standards and Technology (NIST)
9:45 a.m. Break -----
10:00 a.m.

AM Panel: Priority Research Directions for DTEVV

The increasing importance of test and evaluation has bred various lines of T&E-driven research, often occurring in parallel with specific technology development. From robotics to autonomous systems, T&E/V&V practices have had to evolve to ensure that they capably and rigorously assess emergent technologies. This panel will dive into some of these T&E research activities and the driving technologies, along with the practical insight that they have gained in their efforts. Panelists will be further engaged on successes (even early ones), best practices, and challenges.

Moderator
Brian A. Weiss
Research Engineer
University of Maryland Applied Research Laboratory for Intelligence and Security (ARLIS)

Panelists
George Hwang
Chief Technologist, AI/Autonomy Division, Advanced Development Department
Naval Air Warfare Center Aircraft Division (NAWCAD)

Craig Lennon
Scientist
Army Research Lab

Oleg V. Sokolsky
Program Director, Computer and Information Science and Engineering (CISE)
U.S. National Science Foundation (NSF)

Possible additional panelists to be announced
11:00 a.m. Interactive Activity -----
12:15 p.m. Break -----
12:30 p.m.

Lunch and PM Keynote (TBA)

Lunch will be provided

 Stephen Cricchi
Executive Director
Naval Air Warfare Center Aircraft Division (NAWCAD)
1:15 p.m. Break -----
1:30 p.m.

PM Panel: Immediate DTEVV Workforce Needs

T&E, and the overall DTEVV community, is only as effective as their workforce capability. Some personnel enter the workforce with key T&E skills that they built through their undergraduate/graduate education, or through internships. Conversely, some practitioners enter the workforce with adjacent skillsets where they dramatically build their T&E prowess through "on-the-job" training and "getting their hands dirty" in actual T&E efforts. Neither is better than the other - it is often that both provide complementary skills that ultimately boost the finished product, which is the successful development and assessment of an emergent technology. This panel will explore the critical DTEVV skills, either built through formal education or in the workplace, that have shaped the DTEVV community. Likewise, this panel will explore the expected skills that will be needed to develop, implement, and assess the technologies of tomorrow.

Moderator
Donald "Bucket" Costello III
Director of Test and Evaluation of Autonomous Systems
University of Maryland MATRIX Lab

Panelists
Mauricio Castillo-Effen
Fellow
Lockheed Martin Advanced Technologies Laboratories

Wayne Dumais
Deputy Director, Test & Evaluation Division, Science and Technology Directorate
U.S. Department of Homeland Security

CAPT Daniel "Tonto" Kitts, USN
Commodore, Naval Test Wing Atlantic
Naval Air Warfare Center Aircraft Division (NAWCAD)

Theresa Shafer
Chief Technology Officer
NAWCAD
2:30 p.m. Interactive Activity -----
3:45 p.m. Wrapup Donald "Bucket" Costello III
Director of Test and Evaluation of Autonomous Systems
University of Maryland MATRIX Lab
4:30 p.m. Networking -----
5:30 p.m. Evening@SMART Alireza Khaligh
Interim Director
University of Maryland Institute for Systems Research

 

Headshot of Don CostelloDonald "Bucket" Costello III
Director of Test and Evaluation of Autonomous Systems
University of Maryland MATRIX Lab

Brian A. Weiss
Research Engineer
University of Maryland Applied Research Laboratory for Intelligence and Security (ARLIS)

Mauricio Castillo-Effen
Fellow
Lockheed Martin Advanced Technologies Laboratories

 Alireza Khaligh
Interim Director
University of Maryland Institute for Systems Research
Stephen Cricchi
Executive Director
Naval Air Warfare Center Aircraft Division (NAWCAD)		 CAPT Daniel "Tonto" Kitts, USN
Commodore, Naval Test Wing Atlantic
Naval Air Warfare Center Aircraft Division (NAWCAD)

Headshot of Wayne Dumais, smiling and wearing black frame glasses, a dark suit jacket, and a tan tieWayne Dumais
Deputy Director, Test & Evaluation Division, Science and Technology Directorate
U.S. Department of Homeland Security

Headshot of Craig Lennon, a man with short hair wearing glasses, a tan blazer, a blue shirt, and a gold tieCraig Lennon
Scientist
Army Research Lab
Reza Ghodssi
Executive Director of Research and Innovation
Distinguished University Professor
University of Maryland MATRIX Lab		 Kamel "Kam" Saidi
Program Manger and Group Leader, Intelligent Systems Division (ISD)
National Institute of Standards and Technology (NIST)
Samuel Graham, Jr.
Dean, A. James Clark School of Engineering
University of Maryland		 Oleg V. Sokolsky
Program Director, Computer and Information Science and Engineering (CISE)
U.S. National Science Foundation (NSF)
George Hwang
Chief Technologist, AI/Autonomy Division, Advanced Development Department
Naval Air Warfare Center Aircraft Division (NAWCAD)		  

 

Location and Directions

USMSM SMART Building
44219 Airport Road
California, MD 20619

The MATRIX Lab has put together PDF directions to USMSM and the SMART Building for those visiting from Annapolis, Md.; from Virginia via the 301 Harry Nice Bridge, and from the I-495 Washington Beltway.

You also can view the location and get directions from your location on Google Maps.

Ample, free parking is available onsite at the SMART Building (Building 3). There are also parking spots around Buildings 1 and 2 on the USMSM campus.

Hotel Accommodations

The listed establishments have worked with the MATRIX Lab in the past.

Home2 Suites by Hilton
(Preferred - Book MATRIX Rate Here)
301-866-1416
46058 Valley Drive
Lexington Park, MD 20653
6.3 mile drive from the USMSM campus

Hampton Inn
301-863-3200
22211 Three Notch Road
Lexington Park, MD 20653
5.9 mile drive from the USMSM campus

La Quinta Inn & Suites by Wyndham
301-678-9847
22769 Three Notch Road
California, MD 20619
3.8 mile drive from the USMSM campus

Extended Stay America
240-725-0100
46565 Expedition Park Drive
Lexington Park, MD 20653
5.4 mile drive from the USMSM campus

Holiday Inn Express & Suites
240-237-1444
45260 Abell House Lane
California, MD 20619
2.6 mile drive from the USMSM campus

 TownePlace Suites
301-863-1111
22520 Three Notch Road
Lexington Park, MD 20653
4.6 mile drive from the USMSM campus
Fairfield Inn
301-863-0203
22119 Three Notch Road
Lexington Park, MD 20653
6.4 mile drive from the USMSM campus		 The Inn at Leonardtown
301-475-0187
41655 Park Avenue
Leonardtown, MD 20650
8.6 mile drive from the USMSM campus		  

 


Top