HLS - 3. HLS player client

클라이언트쪽은 코드가 상대적으로 많을예정인데

 

일단 영상 플레이어 자체가 이런저런 기능이 들어가다보니 상대적으로 많아진것도 있습니다

 

먼저 api 구현부분부터 보면

 

services/api.ts

export default class Api {
  private instance: AxiosInstance;

  constructor(
    baseURL = "/",
    timeout = 60000
  ) {
    this.instance = axios.create({
      baseURL,
      timeout,
    });

    this.instance.interceptors.request.use(
      (config) => {
        return config;
      },
      (error: AxiosError) => {
        return Promise.reject(error);
      }
    );

    this.instance.interceptors.response.use(
      (response) => {
        return response;
      },
      async (error: AxiosError) => {
        return Promise.reject(error)
      }
    );
  }

  /** - params setting : get(url, {a:1,b:2}) => url?a=1&b=2 */
  protected async get<T>(url: string, params?: any) {
    const res = await this.instance.get<T>(url, { params });

    const rm = res.data;

    return rm;
  }

  protected async post<T>(url: string, data: any, config?: AxiosRequestConfig) {
    const res = await this.instance.post<T>(url, data, config);

    const rm = res.data;

    return rm;
  }
}

언제나 사용하는 공통 axios 셋팅.. 여기에 인증인가 부분이랑 re issue부분은 생략했습니다

 

services/videoApi.ts

class VideoApi extends Api {
  async getVideoList() {
    return await super.get<string[]>("/api/video/list");
  }

  async uploadVideo(
    file: File,
    onUploadProgress?: (e: AxiosProgressEvent) => void
  ) {
    const formData = new FormData();
    formData.append("file", file);

    const res = await super.post<{ msg: string; url: string }>(
      "/api/video/upload",
      formData,
      {
        headers: { "Content-Type": "multipart/form-data" },
        onUploadProgress,
      }
    );

    return res.msg;
  }
}

export const getVideoList = async () => {
  const api = new VideoApi();

  return await api.getVideoList();
};

export const uploadVideo = async (model: {
  file: File;
  onUploadProgress?: (e: AxiosProgressEvent) => void;
}) => {
  const api = new VideoApi();

  return await api.uploadVideo(model.file, model.onUploadProgress);
};

요즘 tanstack query를 이용하고 있어서 선언부 클래스 따로, 사용하는 함수로 정의하고 있습니다

 

예제에 사용하는 api는 2개로

1. video 리스트 조회

2. 비디오 업로드

 

route/upload/index.tsx

const UploadPage = () => {
  const controller = useUploadController();

  return (
    <div className={styles.div_container}>
      <label
        className={`${styles.label_file} ${controller.isIn ? styles.in : ""} ${controller.isLoading ? styles.pending : ""}`}
        onClick={controller.onClickPrevent}
        onDragOver={controller.onDragEnter}
        onDragLeave={controller.onDragLeave}
        onDrop={controller.onDropFiles}>
        <LabelComponent.Loading when={controller.isLoading}>
          <Loading centerFloat={false} />
        </LabelComponent.Loading>
        <LabelComponent.Indicator when={controller.percent !== 0}>
          <div>uploading - {controller.percent}%</div>
        </LabelComponent.Indicator>
        <LabelComponent.Text
          when={!controller.isLoading && controller.percent === 0}>
          <div className={styles.div_upload_btn}>
            Click here to upload video file
            <img
              className={styles.img_upload}
              width={30}
              height={30}
              src="/assets/images/upload.svg"
              onClick={controller.onOpenExplorer}
            />
          </div>
          <p>Or</p>
          <div>Video file in box</div>
          {controller.errMsg && (
            <p className={styles.p_error}>{controller.errMsg}</p>
          )}
        </LabelComponent.Text>
      </label>
      <input
        ref={controller.ref}
        className="hidden"
        type="file"
        onChange={controller.onChangeFile}
      />
    </div>
  );
};

 

hooks/useUploadController.ts

const useUploadController = () => {
  const ref = useRef<HTMLInputElement>(null);

  const [isIn, setIsIn] = useState<boolean>(false);
  const [errMsg, setErrMsg] = useState<string | null>(null);
  const [percent, setPercent] = useState<number>(0);

  const mutation = useMutation({
    mutationFn: uploadVideo,
    onSuccess(result) {
      toast.success(result);
    },
    onError(error) {
      toast.error(error.message);
      setPercent(0);
      ref.current!.value = "";
    },
  });

  const onDropFiles = (e: DragEvent<HTMLLabelElement>) => {
    e.preventDefault();

    const fileList = e.dataTransfer.files;
    const file = _validateFile(fileList);

    if (file) {
      onUploadFile(file);
    }

    onDragLeave();
  };

  const onChangeFile = (e: ChangeEvent<HTMLInputElement>) => {
    const files = e.target.files;

    const file = _validateFile(files);

    console.log({ file });

    if (file) {
      onUploadFile(file);
    }
  };

  const onUploadFile = (file: File) => {
    try {
      mutation.mutate({ file, onUploadProgress });
    } catch (e) {
      ref.current!.value = "";
      setErrMsg(`${e}`);
    }
  };

  const checkFileSize = (file: File) => {
    if (file.size > MAX_FILE_SIZE) {
      const max = Math.round(MAX_FILE_SIZE / 1024 / 1024);
      const uploaded = Math.round(file.size / 1024 / 1024);
      return `File size is too big. Maximum size is ${max}Mb, uploaded file size is ${uploaded}Mb`;
    }

    return null;
  };

  const onUploadProgress = (e: AxiosProgressEvent) => {
    const uploadPercent = Math.round((e.loaded * 100) / (e.total ?? 1));

    if (uploadPercent < 100) {
      setPercent(uploadPercent);
    } else {
      setTimeout(() => {
        ref.current!.value = "";
        setPercent(0);
      }, 3000);
    }
  };

  const _validateFile = (list: FileList | null) => {
    if (list === null) {
      setErrMsg("File doesn't exist");
      return;
    }

    if (list.length > 1) {
      setErrMsg("Please drop only one zip file");
      return;
    }

    const file = list[0];

    if (!file.type.startsWith("video/")) {
      setErrMsg("Only video files are allowed.");
      ref.current!.value = "";

      return;
    } else {
      const msg = checkFileSize(file);

      setErrMsg(msg);
    }

    return file;
  };

  const onOpenExplorer = () => {
    ref.current?.click();
  };

  const onClickPrevent = (e: MouseEvent) => {
    e.preventDefault();
  };

  const onDragEnter = (e: MouseEvent) => {
    onClickPrevent(e);
    if (!isIn) {
      setIsIn(true);
    }
  };

  const onDragLeave = () => {
    if (isIn) {
      setIsIn(false);
    }
  };

  return {
    ref,
    onDropFiles,
    onChangeFile,
    onOpenExplorer,
    onClickPrevent,
    onDragEnter,
    onDragLeave,
    isIn,
    errMsg,
    percent,
    isLoading: mutation.isPending
  };
};

(css를 생략해도 view 코드가 참 많은거 같습니다..)

 

간단하게 input 제어하는부분이랑 서버에 업로드 하는 부분만 구현해놨고

 

서브 컴포넌트를 통해, 어떤 상태일 떄 보여지는지 구분을 해봤는데 조금 더 가독성이 나은거 같습니다

 

추가로 업로드 완료시 toast호출로 표시되는부분도 만들어봤습니다

 

업로드 시간 표시는 용량 작은거로 테스트할때는 거의 %가 안보이고

 

서버 response pending상태로 들어가게 됩니다

 

작은 동영상 파일도 FFmpeg로 변환시 생각보다 오래 걸립니다

 

route/viewer/index.tsx

const ViewerPage = () => {
  const controller = useVideoController();

  return (
    <div>
      {controller.videoError !== undefined ? (
        <p className={styles.p_error}>{controller.videoError}</p>
      ) : (
        <div
          ref={controller.container}
          className={
            controller.isFullScreen
              ? `${styles.div_container} ${styles.full_screen}`
              : styles.div_container
          }>
          {controller.loading && <Loading />}
          <video ref={controller.videoRef} onClick={controller.togglePlay} />
          <Overlay
            isPlaying={controller.isPlaying}
            saveMode={controller.saveMode.start}
            startFlag={controller.startFlag.current}
          />
          <Control
            {...controller}
            saveMode={controller.saveMode.start}
            onChangeTime={controller.seek}
          />
          <VideoList
            {...controller}
            onChangeTime={controller.seek}
            onChangeAutoPlay={controller.setAutoPlay}
            onClickVideo={controller.setVideoIdx}
          />
        </div>
      )}
    </div>
  );
};

 

container ref부분은 전체화면 처리 용도로 사용하고 있고

 

Overlay는 좌우 이동, 타임라인 저장시 화면에 표시 등을 위해 만들어봤고

 

Control은 재생/정지 토글버튼, 음량, 비디오 앞으로/뒤로, 타임라인 저장, 전체화면, 화질 조정 등이 있습니다

 

VideoList는 영상들, 저장된 타임라인을 보여주는 부분으로 구성해봤습니다

 

outro에서 github 링크부분에서 확인해주시면 될거 같습니다

(전부다 넣기에는 너무 많아서...)

 

hooks/useVideoController.ts

const useVideoController = () => {
  const { data } = useQuery({
    queryKey: ["video_list"],
    queryFn: getVideoList,
  });

  const videoList = data ?? [];

  const videoRef = useRef<HTMLVideoElement>(null);

  /** - video, control element */
  const container = useRef<HTMLDivElement>(null);

  /** - hls */
  const hlsRef = useRef<Hls>(null);

  /** - throttle */
  const isCalled = useRef<boolean>(false);

  /** - to prevent auto play */
  const startFlag = useRef<boolean>(false);

  /** - video quality levels */
  const [levelList, setLevelList] = useState<Level[]>([]);

  /** - video total and current duration */
  const [duration, setDuration] = useState<DurationModel>({
    current: 0,
    total: 0,
  });

  /** - loading for video */
  const [loading, setLoading] = useState<boolean>(false);

  /** - selected video idx */
  const [videoIdx, setVideoIdx] = useState<number>(0);

  /** - quality level idx */
  const [levelIdx, setLevelIdx] = useState<number>(0);

  /** - to toggle button */
  const [isPlaying, setIsPlaying] = useState<boolean>(false);

  /** - check screen mode */
  const [isFullScreen, setIsFullScreen] = useState<boolean>(false);

  /** - video error msg */
  const [videoError, setVideoError] = useState<string>();

  /** - play next video automatically */
  const [autoPlay, setAutoPlay] = useState<boolean>(true);

  /** - saved time line */
  const [savedTimes, setSavedTimes] = useState<TimelineModel[]>([]);

  /** - save mode on/off */
  const [saveMode, setSaveMode] = useState<SaveTimeModel>({
    start: false,
    startTime: 0,
    endTime: 0,
  });

  // init media and listener
  useEffect(() => {
    if (videoList.length > 0) {
      _setVideoListener();
      _initMedia();
    }

    return () => {
      _setVideoListener(true);
      _reset();
    };
  }, [videoList, videoIdx]);

  // screen mode change
  useEffect(() => {
    document.onfullscreenchange = () => {
      setIsFullScreen(!isFullScreen);
    };

    return () => {
      document.onfullscreenchange = null;
    };
  }, [isFullScreen]);

  // save timeline
  useEffect(() => {
    if (saveMode.startTime < saveMode.endTime) {
      const { startTime, endTime, img } = saveMode;

      setSavedTimes([...savedTimes, { startTime, endTime, img }]);
    }
  }, [saveMode]);

  // space : play/pause toggle, left/right arrow : move video time
  useEffect(() => {
    document.addEventListener("keydown", _onKeyDown);

    return () => {
      document.removeEventListener("keydown", _onKeyDown);
    };
  }, [isPlaying, duration.current]);

  /** - set video url */
  const _initMedia = async () => {
    setLoading(true);

    const url = `/api/video/${videoList[videoIdx]}/master.m3u8`;

    console.log({ url, videoList });

    if (Hls.isSupported()) {
      hlsRef.current = new Hls();
      hlsRef.current.loadSource(url);
      hlsRef.current.attachMedia(videoRef.current!);

      hlsRef.current.on(Events.MANIFEST_PARSED, (_, data) => {
        console.log({ data });
        setLevelIdx(data.firstLevel);
        setLevelList(data.levels);
      });
    } else {
      toast.error("This browser doesn't support HLS");
      videoRef.current!.src = url;
    }
  };

  /** - set video listeners */
  const _setVideoListener = (reset = false) => {
    if (reset) {
      if(videoRef.current){
        videoRef.current.onloadedmetadata = null;
        videoRef.current.oncanplay = null;
        videoRef.current.ontimeupdate = null;
        videoRef.current.onended = null;
        videoRef.current.onplay = null;
        videoRef.current.onpause = null;
        videoRef.current.onerror = null;
      }
    } else {
      videoRef.current!.onloadedmetadata = _onLoadMetaData;
      videoRef.current!.oncanplay = _canPlayVideo;
      videoRef.current!.ontimeupdate = _onPlayingTimeUpdate;
      videoRef.current!.onended = onNextVideo;
      videoRef.current!.onplay = () => {
        setIsPlaying(true);
      };
      videoRef.current!.onpause = () => {
        setIsPlaying(false);
      };
      videoRef.current!.onerror = (e) => {
        setVideoError(`[Error] ${JSON.stringify(e)}`);
      };
    }
  };

  /** - setting duration */
  const _onLoadMetaData = (e: Event) => {
    const target = e.target as HTMLVideoElement;

    setDuration((state) => {
      return { ...state, total: target.duration };
    });

    if (startFlag.current && autoPlay) {
      _play();
    }
  };

  /** - check if video is playable */
  const _canPlayVideo = () => {
    if (videoRef.current!.readyState === 4) {
      setLoading(false);
    } else {
      setLoading(true);
    }
  };

  /** - move to the next video */
  const onNextVideo = () => {
    const nextIdx = videoIdx + 1;

    if (nextIdx < videoList.length) {
      setVideoIdx(nextIdx);
    } else {
      setVideoIdx(0);
    }
  };

  /** - move to previous video */
  const onPreviousVideo = () => {
    const previousIdx = videoIdx - 1;

    if (previousIdx >= 0) {
      setVideoIdx(previousIdx);
    } else {
      setVideoIdx(videoList.length - 1);
    }
  };

  /** - update playing time */
  const _onPlayingTimeUpdate = (e: Event) => {
    // when level is changed, stop updating
    if (videoRef.current?.paused) {
      return;
    }

    const target = e.target as HTMLVideoElement;

    setDuration((state) => {
      return { ...state, current: target.currentTime };
    });
  };

  /** - reset video data and remove listener, saved times */
  const _reset = () => {
    hlsRef.current?.destroy();

    setLevelList([]);
    setSavedTimes([]);
    setSaveMode({ start: false, startTime: 0, endTime: 0 });
    setDuration({ current: 0, total: 0 });
  };

  /** - keyboard event */
  const _onKeyDown = (e: KeyboardEvent) => {
    switch (e.code) {
      case KEY_CODE.space:
        togglePlay();
        break;
      case KEY_CODE.arrowRight:
        seek(duration.current + 1);
        break;
      case KEY_CODE.arrowLeft:
        seek(duration.current - 1);
        break;
    }
  };

  /** - play video */
  const _play = async () => {
    if (videoRef.current!.paused) {
      await videoRef.current?.play();
    }
  };

  /** - pause video */
  const _pause = () => {
    if (!videoRef.current!.paused) {
      videoRef.current?.pause();
    }
  };

  /** - move to position */
  const seek = (time: number) => {
    if (isCalled.current) {
      return;
    } else {
      isCalled.current = true;

      commonUtil.delay(100).then(() => {
        isCalled.current = false;
      });
    }

    videoRef.current!.currentTime = time;

    setLoading(true);

    setDuration((state) => {
      return { ...state, current: time };
    });
  };

  /** - change volume */
  const onChangeVolume = (value: number) => {
    videoRef.current!.volume = value;
  };

  /** - toggle save time mode */
  const toggleSaveMode = async () => {
    let img: string | undefined;
    if (!saveMode.start) {
      img = await _getThumbnail();
      console.log({ img });
    }

    setSaveMode((state) => {
      if (state.start) {
        return { ...state, start: !state.start, endTime: duration.current };
      } else {
        return {
          ...state,
          start: !state.start,
          startTime: duration.current,
          img,
        };
      }
    });
  };

  /** - toggle play/pause */
  const togglePlay = () => {
    if (isPlaying) {
      _pause();
    } else {
      if (!startFlag.current) {
        startFlag.current = true;
      }
      _play();
    }
  };

  /** - toggle save time mode */
  const toggleSaveMode = async () => {
    let img: string | undefined;
    if (!saveMode.start) {
      img = await _getThumbnail();
      console.log({ img });
    }

    setSaveMode((state) => {
      if (state.start) {
        state.endTime = duration.current;
      } else {
        state.startTime = duration.current;
        state.img = img;
      }

      return { ...state, start: !state.start };
    });
  };

  /** - get thumbnail from video */
  const _getThumbnail = async () => {
    const canvas = await html2canvas(videoRef.current!, { scale: 0.1 });
    const imageFile = canvas.toDataURL("image/jpg", 0.1);

    return imageFile;
  };

  /** - change quality level */
  const onChangeLevel = (i: number) => {
    setLevelIdx(i);

    hlsRef.current!.currentLevel = i;

    if (startFlag.current) {
      hlsRef.current!.detachMedia();

      hlsRef.current!.attachMedia(videoRef.current!);

      setLoading(true);
      _pause();
    }
  };

  return {
    container,
    videoRef,
    levelList,
    levelIdx,
    videoList,
    videoIdx,
    setVideoIdx,
    duration,
    isFullScreen,
    toggleScreenMode,
    togglePlay,
    toggleSaveMode,
    saveMode,
    savedTimes,
    seek,
    loading,
    onChangeVolume,
    isPlaying,
    onNextVideo,
    onPreviousVideo,
    videoError,
    autoPlay,
    setAutoPlay,
    onChangeLevel,
    startFlag,
  };
};

export default useVideoController;

편의 기능들이 많이 있어서 해당 부분에서는 HLS 사용 부분

 

그리고 thumbnail생성 부분만 다뤄봐도 될거 같습니다

 

먼저 setVideoListener부터 보면, HLS관련 이벤트 설정/해제 부분으로

 

영상 시간 셋팅, play/stop/update 등 영상 시간과 관련된 부분과 실행 가능여부 콜백들을 설정해줍니다

 

다음 initMedia부분을 보면

 

HLS에 master.m3u8을 조회하고, level(화질 리스트)를 설정해주면

 

attachMedia로 설정된 video 태그에 영상이 전달되게 됩니다

 

네트워크에 따른 화질을 자동으로 셋팅하고 싶으면 다음과 같게 하면 됩니다

const hls = new Hls({
  startLevel: -1, // -1이면 가장 적절한 화질을 자동 선택
});

그리고 영상을 변경할 때는 onChangeLevel쪽을 보면 되는데

 

먼저 HLS 레벨 설정, detachMedia, attachMedia순으로 콜 해주면 됩니다

 

다음으로 특정 타임라인 저장하는 부분을 보면

 

캡쳐하는 duration 저장 및 html2canvs라이브러리를 이용해서 해당 돔을 이미지로 추출합니다

 

그 후 timeline과 같이 저장해줍니다

 

나머지는 간단한 키보드 이벤트, duration 계산 기타 등등이니 넘어가도록 하겠습니다

 

Intro : https://devmemory.tistory.com/130

Node server : https://devmemory.tistory.com/131

Outro : https://devmemory.tistory.com/133