James Duffy was born on May 31, 1918, in Brooklyn, New York. He enlisted in the U.S. Army on September 20, 1940, and served in the Field Artillery until being discharged on September 19, 1941. Duffy then enlisted in the Aviation Cadet Program of the U.S. Army Air Forces on January 31, 1942, and was commissioned a 2d Lt and awarded his pilot wings at Spence Field, Georgia, on January 14, 1943. After completing P-47 Thunderbolt training, he joined the 354th Fighter Squadron of the 355th Fighter Group and served in England from July 1943 to July 1945. During this time, Capt Duffy flew 101 combat missions and was credited with destroying 5.2 enemy aircraft in aerial combat in P-47's and P-51 Mustangs, and another 9 aircraft on the ground while strafing enemy airfields. He served on occupation duty in Germany from July 1945 to May 1946, and then returned to the U.S. in June 1946. During his occupation duty, Duffy served as Commander of the 354th Fighter Squadron from September 1945 to April 1946. After serving briefly at Fort Dix, New Jersey, with Headquarters Tactical Air Command at Langley Field, Virginia, and at Greenville Army Air Base, South Carolina, he left active duty on January 5, 1947. Duffy remained in the Air Force Reserve until being discharged on April 1, 1955. He died on September 10, 1994.

His 2nd (of 4) Distinguished Flying Cross Citation reads:

For extraordinary achievement and heroism in aerial combat and the destruction of five enemy airplanes over enemy occupied Continental Europe. The skillful and zealous manner in which Captain Duffy has sought out the enemy and destroyed him, and his devotion to duty and courage under all conditions served as an inspiration to his fellow flyers. His actions on all these occasions reflect the highest credit upon himself and the Armed Forces of the United States.