tedshd's DevNote

Develop & Design Note by Ted

JavaScript - 半形 全形 轉換

JavaScript - 半形 全形 轉換

因為一個奇怪的需求?

需要把使用者輸入的東西轉成可以方便驗證的格式(這應該後端做吧?)

反正因為以前也做過半形轉成全形(痛苦的回憶, 為啥政府的 opendata 各縣市格式不一致就算了, 竟然還有全形半形的問題...)

這邊就簡單的只針對最前面的基本拉丁字母字元符號集作轉換

因為大部分的需求應該這樣就可以應付了?

但是空白字元要特殊處理

那接下來只要知道需要轉換的字元範圍即可

全形從 U+FF01U+FF5E

半形從 U+0021U+007E

把 16 進制轉換回來就是

65281 到 65374

33 到 126

上述怎麼轉的?

就是這樣

parseInt('0xFF01', 16)

只要判斷輸入的字元這這些編碼區間就進行轉換

trans char

所以簡單做了一個 demo 頁面

程式檢視原始碼即可

Refer - Wiki

Refer - Unicode / UTF-8 字元編碼區間表 - 2013

Chrome headless 研究筆記

Chrome headless 研究筆記

最近要用到 chrome headless 的部分功能

且得在 ubuntu or debian server 實踐

所以筆記一下

Intro

headless 大致分兩種用法

  1. CLI

  2. libary

Refer - Getting Started with Headless Chrome

CLI 用法簡單但是受限很多且沒有找到具體詳細完整的文件, 就連用 man 查都查不到...

也有可能是我自己沒有認真找

libary 就相對容易, 連 Google 自己都有提供部分語言的版本, 且有前端工程師很熟的 Node 版本

on ubuntu server

sudo apt-get install chromium-browser

ex:

chromium-browser --headless --screenshot=test.png --disable-gpu --window-size=320,480 --user-agent="Mozilla/5.0 (iPhone; CPU iPhone OS 7_1_2 like Mac OS X) AppleWebKit/537.51.2 (KHTML, like Gecko) Version/7.0 Mobile/11D257 Safari/9537.53" http://tedshd.logdown.com

這指令下了截圖, 設定寬高, 模擬 UserAgent

當然也有 --dump-dom 可以看 source code

使用 puppeteer 處理

puppeteer

puppeteer 是 Google 維護的用 node 處理 headless 的套件

用了一下真是感到清爽好用

放一段最簡單的 smaple code

const puppeteer = require('puppeteer');

(async () => {
    const browser = await puppeteer.launch();
    const page = await browser.newPage();
    page.setUserAgent('Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:64.0) Gecko/20100101 Firefox/64.0');
        await page.goto('https://tysh310246.blogspot.com');

    await page.screenshot({path: 'example.png', fullPage: true});
    await browser.close();

})();

就是如此清爽

且有很多包好的 function 可以用

看了一下文件根本可以用 headless 取代 selenium + webdriver.io 了

但是目前的需求也不需要用到太多其他的功能

有空再慢慢研究

Puppeteer API Tip-Of-Tree

字型

在一台 server 上除非要截的網站是用 webfont 不然就會發現截的圖因為沒有字型所以中文會是方塊字

這部分就得處理一下

可以直接裝 Google 的 noto 字型

wget -c https://noto-website.storage.googleapis.com/pkgs/Noto-hinted.zip

把字型檔載下來放在

/usr/local/share/fonts/

設定 owner

設定權限

安裝字型

sudo fc-cache -fv

搞定

這邊建議直接裝全部語系的字型較為完整, 以免會遇到其他語系還是會有問題

Refer - Ubuntu環境下,手動安裝思源字型

2018 Google cloud 訓練營 筆記

2018 Google cloud 訓練營 筆記

Become a Google Cloud Platform expert with hands-on training. GCP Essentials

Gcloud cli

PROJECT_ID:

gcloud auth list

gcloud config list project

BASIC

gcloud -h

gcloud config --help || gcloud help config

gcloud config list --all

Managing Cloud Storage data

Try creating a Cloud Storage bucket. Bucket names must be unique, so replace unique-name with something else, or append the name to make it unique.

gsutil mb gs://<unique-name>

Upload file to bucket

gsutil cp <file> gs://<unique-name>

Create a new persistent disk

Because we want to attach this disk to the virtual machine instance we created in the previous step, the zone must be the same.

gcloud compute disks create <mydisk name> --size=200GB \ --zone us-central1-c

Attaching a disk

Attaching the persistent disk

You can attach a disk to a running virtual machine. Let's attach the new disk (mydisk) to the virtual machine instance you just created (gcelab).

gcloud compute instances attach-disk --disk --zone us-central1-c

Finding the persistent disk in the virtual machine

The persistent disk is now available as a block device in the virtual machine instance. Let's take a look.

go to this instance

ls -l /dev/disk/by-id/

You found the file, the default name is:

scsi-0Google_PersistentDisk_persistent-disk-1.

If you want a different device name, when you attach the disk, you would specify the device-name parameter. For example, to specify a device name, when you attach the disk you would use the command:

gcloud compute instances attach-disk gcelab --disk mydisk --device-name <YOUR_DEVICE_NAME> --zone us-central1-c

Formatting and mounting the persistent disk

Once you find the block device, you can partition the disk, format it, and then mount it using the following Linux utilities:

  • mkfs: creates a filesystem
  • mount: attaches to a filesystem

Make a mount point:

sudo mkdir /mnt/mydisk

Next, format the disk with a single ext4 filesystem using the mkfs tool. This command deletes all data from the specified disk:

sudo mkfs.ext4 -F -E lazy_itable_init=0,lazy_journal_init=0,discard /dev/disk/by-id/scsi-0Google_PersistentDisk_persistent-disk-1

Last lines of the output.

Allocating group tables: done
Writing inode tables: done
Creating journal (262144 blocks): done
Writing superblocks and filesystem accounting information: done

Now use the mount tool to mount the disk to the instance with the discard option enabled:

sudo mount -o discard,defaults /dev/disk/by-id/scsi-0Google_PersistentDisk_persistent-disk-1 /mnt/mydisk

Q: Can you prevent the destruction of an attached persistent disk when the instance is deleted?

A: Yes, deselect the option Delete boot disk when instance is deleted when creating an instance

A:Yes, use the –keep-disks option with the gcloud compute instances delete command

Q: For migrating data from a persistent disk to another region, reorder the following steps in which they should be performed:

  1. Unmount file system(s)
  2. Create snapshot
  3. Create disk
  4. Create instance
  5. Attach disk

Hello Node Kubernetes

push docker image to instance

gcloud docker -- push gcr.io/<PROJECT_ID>/<hello-node>:<v1>

Create your cluster

gcloud config set project <PROJECT_ID>

Create a cluster with two n1-standard-1 nodes (this will take a few minutes to complete):

gcloud container clusters create <cluster name> \
                --num-nodes 2 \
                --machine-type n1-standard-1 \
                --zone us-central1-a

Note: You can also create this cluster through the Console, image shown above: Kubernetes Engine > Kubernetes clusters > Create cluster.

It is recommended to create the cluster in the same zone as the storage bucket used by the container registry (see previous step).

Create your pod

A Kubernetes pod is a group of containers tied together for administration and networking purposes. It can contain single or multiple containers. Here you'll use one container built with your Node.js image stored in your private container registry. It will serve content on port 8080.

Create a pod with the kubectl run command (replace PROJECT_ID with your GCP Project ID, found in the console and in the Connection Details section of the lab):

kubectl run <hello-node> \
    --image=gcr.io/<PROJECT_ID>/<hello-node>:v1 \
    --port=8080

hello-node is container image name

To view the deployment, run:

kubectl get deployments

To view the pod created by the deployment, run:

kubectl get pods
kubectl cluster-info

Overview of kubectl

And for troubleshooting :

kubectl get events

kubectl logs <pod-name>

Allow external traffic

By default, the pod is only accessible by its internal IP within the cluster. In order to make the hello-node container accessible from outside the Kubernetes virtual network, you have to expose the pod as a Kubernetes service.(預設只有內部 IP 可以讀取)

From Cloud Shell you can expose the pod to the public internet with the kubectl expose command combined with the --type="LoadBalancer" flag. This flag is required for the creation of an externally accessible IP:

kubectl expose deployment <container image name> --type="LoadBalancer"

To find the publicly-accessible IP address of the service, request kubectl to list all the cluster services:

kubectl get services

This is the output you should see:

NAME         CLUSTER-IP     EXTERNAL-IP      PORT(S)    AGE
hello-node   10.3.250.149   104.154.90.147   8080/TCP   1m
kubernetes   10.3.240.1     <none>           443/TCP    5m

open 104.154.90.147:8080

Scale up your service

One of the powerful features offered by Kubernetes is how easy it is to scale your application. Suppose you suddenly need more capacity for your application. You can tell the replication controller to manage a new number of replicas for your pod:

kubectl scale deployment <hello-node> --replicas=4

You can request a description of the updated deployment:

kubectl get deployment

You can also list the all pods:

kubectl get pods

Roll out an upgrade to your service

At some point the application that you've deployed to production will require bug fixes or additional features. Kubernetes helps you deploy a new version to production without impacting your users.

Modify files & rebuild & push

docker build -t gcr.io/<PROJECT_ID>/<hello-node>:v2 .
gcloud docker -- push gcr.io/<PROJECT_ID>/<hello-node>:v2

New version image

To do this, use the kubectl edit command. It opens a text editor displaying the full deployment yaml configuration. It isn't necessary to understand the full yaml config right now, just understand that by updating the spec.template.spec.containers.image field in the config you are telling the deployment to update the pods with the new image.

kubectl edit deployment <hello-node>

Modify image & save

Run the following to update the deployment with the new image. New pods will be created with the new image and the old pods will be deleted.

kubectl get deployments

While this is happening, the users of your services shouldn't see any interruption. After a little while they'll start accessing the new version of your application. You can find more details on rolling updates in this documentation.

Hopefully with these deployment, scaling, and updated features, once you've set up your Kubernetes Engine cluster, you'll agree that Kubernetes will help you focus on the application rather than the infrastructure.

Set Up Network and HTTP Load Balancers

In this hands-on lab, you'll learn the differences between a network load balancer and a HTTP load balancer, and how to set them up for your applications running on Google Compute Engine virtual machines.

There are several ways you can load balance in Google Cloud Platform. This lab takes you through the setup of the following load balancers.:

Set the default region and zone for all resources

In Cloud Shell, set the default zone:

gcloud config set compute/zone us-central1-a

Set the default region:

gcloud config set compute/region us-central1

Regions & Zones documentation

Create multiple web server instances

To simulate serving from a cluster of machines, create a simple cluster of Nginx web servers to serve static content using Instance Templates and Managed Instance Groups. Instance Templates define the look of every virtual machine in the cluster (disk, CPUs, memory, etc). Managed Instance Groups instantiate a number of virtual machine instances using the Instance Template.

To create the Nginx web server clusters, create the following:

  • A startup script to be used by every virtual machine instance to setup Nginx server upon startup
  • An instance template to use the startup script
  • A target pool
  • A managed instance group using the instance template

Still in Cloud Shell, create a startup script to be used by every virtual machine instance. This script sets up the Nginx server upon startup:

cat << EOF > startup.sh
#! /bin/bash
apt-get update
apt-get install -y nginx
service nginx start
sed -i -- 's/nginx/Google Cloud Platform - '"\$HOSTNAME"'/' /var/www/html/index.nginx-debian.html
EOF

Create an instance template, which uses the startup script:

gcloud compute instance-templates create nginx-template \
         --metadata-from-file startup-script=startup.sh

Create a target pool. A target pool allows a single access point to all the instances in a group and is necessary for load balancing in the future steps.

gcloud compute target-pools create nginx-pool

Create a managed instance group using the instance template:

gcloud compute instance-groups managed create nginx-group \
         --base-instance-name nginx \
         --size 2 \
         --template nginx-template \
         --target-pool nginx-pool \

List instance

gcloud compute instances list

Now configure a firewall so that you can connect to the machines on port 80 via the EXTERNAL_IP addresses:

gcloud compute firewall-rules create www-firewall --allow tcp:80

Create a Network Load Balancer

Network load balancing allows you to balance the load of your systems based on incoming IP protocol data, such as address, port, and protocol type. You also get some options that are not available, with HTTP(S) load balancing. For example, you can load balance additional TCP/UDP-based protocols such as SMTP traffic. And if your application is interested in TCP-connection-related characteristics, network load balancing allows your app to inspect the packets, where HTTP(S) load balancing does not.

For more information, see Setting Up Network Load Balancing.

Create an L3 network load balancer targeting your instance group:

gcloud compute forwarding-rules create nginx-lb \
         --region us-central1 \
         --ports=80 \
         --target-pool nginx-pool

List all Google Compute Engine forwarding rules in your project.

gcloud compute forwarding-rules list

Create a HTTP(s) Load Balancer

HTTP(S) load balancing provides global load balancing for HTTP(S) requests destined for your instances. You can configure URL rules that route some URLs to one set of instances and route other URLs to other instances. Requests are always routed to the instance group that is closest to the user, provided that group has enough capacity and is appropriate for the request. If the closest group does not have enough capacity, the request is sent to the closest group that does have capacity.

Learn more about the HTTP(s) Load Balancer in the documentation.

First, create a health check. Health checks verify that the instance is responding to HTTP or HTTPS traffic:

gcloud compute http-health-checks create http-basic-check

Define an HTTP service and map a port name to the relevant port for the instance group. Now the load balancing service can forward traffic to the named port:

gcloud compute instance-groups managed \
       set-named-ports nginx-group \
       --named-ports http:80

Create a backend service:

gcloud compute backend-services create nginx-backend \
      --protocol HTTP --http-health-checks http-basic-check --global

Add the instance group into the backend service:

gcloud compute backend-services add-backend nginx-backend \
    --instance-group nginx-group \
    --instance-group-zone us-central1-a \
    --global

Create a default URL map that directs all incoming requests to all your instances:

gcloud compute url-maps create web-map \
    --default-service nginx-backend

To direct traffic to different instances based on the URL being requested, see content-based routing.

Create a target HTTP proxy to route requests to your URL map:

gcloud compute target-http-proxies create http-lb-proxy \
    --url-map web-map

Create a global forwarding rule to handle and route incoming requests. A forwarding rule sends traffic to a specific target HTTP or HTTPS proxy depending on the IP address, IP protocol, and port specified. The global forwarding rule does not support multiple ports.

gcloud compute forwarding-rules create http-content-rule \
        --global \
        --target-http-proxy http-lb-proxy \
        --ports 80

After creating the global forwarding rule, it can take several minutes for your configuration to propagate.

gcloud compute forwarding-rules list

Finally

GCP Essentials

JavaScript - 計算網頁元件的曝光

JavaScript - 計算網頁元件的曝光

因為某些需求需要精確計算某些網頁上的 UI 元件的曝光

沒錯

就是要像廣告計算 impression 一樣

之前只要確認有該元件的 request 就好了

但是為了精確追求計算曝光, 所以必須做到像 Google Ad 一樣

要到該網頁元件讓使用者看到後才能計算曝光

聽起來這要求不太容易

但是實作下去其實發現還好誒!!

概念

DOM 元件進入可視範圍觸發 impression

所以需要能夠釐清一些問題

  1. 如何確認可視範圍?

  2. 如何確認要記錄 impression 的 DOM 元件在可視範圍?

  3. 如何在進入可視範圍時觸發?

看似困難

但是如果熟悉 JavaScript Window 物件和 DOM 物件的話就會發現很好解

實作

如何確認可視範圍?
window.document.documentElement.clientHeight

window.document.documentElement.clientWidth

完美解決

如何確認要記錄 impression 的 DOM 元件在可視範圍?
<DOM Object>.getBoundingClientRect();

完美解決

但是要取到對應可是範圍的位置建議使用(top, left), x y IE 和 Edge 不支援

還有一點要注意就是該 DOM 要先長在 browser 上面且不是 display: none 的狀態才能拿到值

Refer - Element.getBoundingClientRect() - Web API 接口 | MDN

Refer - ClientRect object (Internet Explorer)

如何在進入可視範圍時觸發?

這就得取決於使用情境了

在適當的 Event 去觸發

Ex:

在 list view 時就單純的用 scroll 事件觸發確認是否是進入可視範圍(當然得做 debounce 處理)

在 carousel view 就滑動或點擊觸發轉動時確認是否是進入可視範圍(如果有做動畫效果, 得等到動畫效果結束再去確認)

結論

熟悉基本 API 後就可以很輕鬆地解決很多問題 XD

最後附上包好的套件

GitHub - impression